Feature-based slam with z-axis location

ABSTRACT

Embodiments are disclosed for a feature-based simultaneous localization and mapping (SLAM) system and method that generates radio maps for environments that are not accessible for surveying. More accurate radio maps are generated for an unsurveyed environment by determining a best estimate of a mobile device state from harvested traced data that maximizes a posterior probability of the mobile device state given measurements, landmarks and loop constraints.

CROSS-RELATED APPLICATION

This application claims the benefit of priority of U.S. ProvisionalPatent Application No. 62/679,735, filed Jun. 1, 2018, for“Feature-Based SLAM,” which provisional patent application isincorporated by reference herein in its entirety.

TECHNICAL FIELD

The disclosure generally relates to extending a radio map.

BACKGROUND

Some mobile devices have features for determining a geographic location.For example, a mobile device can include a receiver for receivingsignals from a global satellite system (e.g., global positioning systemor GPS). The mobile device can determine a geographic location,including latitude and longitude, using the received GPS signals. Inmany places where a mobile device does not have a line of sight with GPSsatellites, GPS location determination can be error prone. For example,a conventional mobile device often fails to determine a location ordetermines a location with poor accuracy based on GPS signals when thedevice is inside a building or tunnel. For example, areas withobstructing buildings can diminish line of sight of the GPS signals andintroduce error. In addition, even if a mobile device has lines of sightwith multiple GPS satellites, error margin of GPS location can be in theorder of tens of meters. Such error margin may be too large fordetermining on which floor of a building the mobile device is located,in which room of the floor the mobile device is located, on which sideof a street the mobile device is located, on which block the mobiledevice is located, etc.

SUMMARY

Embodiments are disclosed for a feature-based simultaneous localizationand mapping (SLAM) system and method that generates radio maps forenvironments that are not accessible for surveying. More accurate radiomaps are generated for an unsurveyed environment by determining a bestestimate of a mobile device state from harvested traced data thatmaximizes a posterior probability of the mobile device state givenmeasurements, landmarks and loop constraints.

In an embodiment, a method comprises: receiving, by one or moreprocessors, a first set of trace data from a first mobile deviceoperating in an environment, the first set of trace data describing afirst set of states of a first trajectory of the first mobile device inthe environment, the first set of trace data including a first set ofwireless access point data and a first set of sensor measurementscollected by the first mobile device while operating in the environment,the first set of sensor measurements including pressure measurements;receiving, by the one or more processors, a second set of trace datafrom a second mobile device operating in the environment, the second setof trace data describing a second set of states of a second trajectoryof the second mobile device in the environment, the second set of tracedata including a second set of wireless access point data and a secondset of sensor measurements collected by the second mobile device whileoperating in the environment, the second set of sensor measurementsincluding pressure measurements; determining, by the one or moreprocessors, floor transitions in the first trajectory and the secondtrajectory based on the pressure measurements; dividing, by the one ormore processors, the first and second trajectories into floor nodesbased on the floor transitions; determining, by the one or moreprocessors, a distance between the floor nodes; determining, by the oneor more processors, a floor ordinal based on the distance; comparing, bythe one or more processors, portions of the first and secondtrajectories having the same floor ordinal using an optimization solverand a number of location constraints; and generating, by the one or moreprocessors, a radio map based on results of the comparing.

Other embodiments are directed to systems, apparatuses andnon-transitory, computer-readable mediums.

Particular implementations may provide one or more of the followingadvantages. Feature-based SLAM allows radio maps to be generated forenvironments that are not accessible for surveying. More accurate radiomaps are generated for unsurveyed environments using posteriorprobabilities computed from wireless network access point data andsensor measurements collected from mobile devices operating in theenvironment.

Details of one or more implementations are set forth in the accompanyingdrawings and the description below. Other features, aspects, andpotential advantages will be apparent from the description and drawings,and from the claims.

DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a surveying technique fordetermining positioning.

FIG. 2 shows an example of an RSSI probability distribution graph usedin the surveying technique of FIG. 1.

FIG. 3 shows an example of a radio map for a venue.

FIG. 4 is a block diagram illustrating an exemplary process of extendingthe radio map of FIG. 3 using harvest data.

FIG. 5 is a block diagram of an exemplary localizer for determining anoptimized trajectory based on the harvest data illustrated in FIG. 4.

FIG. 6 is a flowchart of an exemplary process of extending a radio map.

FIG. 7 is a block diagram of another exemplary system for extending aradio map using feature-based SLAM.

FIG. 8A is a flowchart of another exemplary process of extending a radiomap using feature-based SLAM.

FIG. 8B is a flowchart of another exemplary process of extending a radiomap using feature-based SLAM with z-axis location.

FIG. 9 is a block diagram of an exemplary system architecture of anelectronic device implementing the features and operations described inreference to FIGS. 1-8.

FIG. 10 is a block diagram of an exemplary device architecture of acomputing device implementing the features and operations described inreference to FIGS. 1-8.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

Indoor positioning systems can use wireless local-area network (WLAN)(e.g., Wi-Fi) infrastructure to allow a mobile device to determine itsposition in an indoor venue, where other techniques such as GPS may notbe able to provide accurate and/or precise position information. SuchWi-Fi-based positioning systems typically involve at least two phases—adata training phase and a positioning phase. During the data trainingphase (e.g., sometimes referred to as the surveying phase), a mobilesurvey device is positioned at various reference points throughout thevenue. In some implementations, the reference points are predeterminedlocations within the venue for which positioning information is desired.The predetermined locations (e.g., for which the data training phase isperformed) can later be identified as a current location of a devicewhen a subsequent positioning phase is performed on the device. In someimplementations, the actual locations of the reference points may not bepredetermined, but may instead be determined according to one or morerules and/or criteria. For example, a first reference point may bedefined at a particular location of the venue (e.g., at an entrance ofthe venue), and additional reference points may be defined at aparticular distance interval (e.g., every 10 meters) in one or moreparticular directions, as described in more detail below.

An operator of the survey device (e.g., a surveyor) may travel to afirst reference point within the venue and provide an input on a userinterface of the survey device to indicate the position of the firstreference point relative to the venue. For example, the surveyor maydrop a pin on an indoor map representation of the venue. The surveyormay then cause the survey device to gather a plurality of measurements.In particular, the survey device determines all access points (APs)(e.g., wireless APs) that the survey device is in communication with andmeasures the received signal strength indicators (RSSIs) of each of thesignals received from each of the APs. For each reference point, aplurality (e.g., hundreds) of RSSI measurements are obtained for eachAP. Measurements may be obtained at a set interval (e.g., every fewseconds). Measurements may be obtained over multiple days and underdifferent conditions, such as under different climate conditions,different venue conditions (e.g., when the venue is highly populated,slightly populated, and unpopulated), different times of day, and/ordifferent physical venue conditions (e.g., different combinations ofdoors and/or windows within the venue being open or closed, etc.). Thesurveyor may then travel to a second reference point and repeat theprocedure, and so on until a comprehensive number of reference pointswithin the venue have been gathered. The full set of measurements forall APs at all reference points within the venue are stored in adatabase (e.g., a fingerprint database). The collection of measurementsis sometimes referred to as a “location fingerprint” of the venue. Atthis stage, the location data included in the database may be largelysurvey data (e.g., measurements obtained by the survey device during thedata training phase).

In some implementations, the location data may be obtained using one ormore techniques other than the surveying technique described above. Forexample, other source data may be obtained and used to provide thelocation fingerprint of the venue. In general, the location fingerprintis based on source data that is deemed to be high quality and accuratedata (e.g., data that correlates RSSI measurements to correspondingpositions to a relatively high degree of accuracy). Other types ofsource data that can be used to create the location fingerprint (and,e.g., a radio map) are described below.

The positioning phase occurs after the training phase has at leastpartially been completed. During the positioning phase, a mobile device(e.g., a mobile device separate from the survey device) at a particularlocation within the venue may attempt to determine its location. Themobile device performs a scan of all APs in communication range of themobile device and obtains RSSI measurements for signals received fromeach AP. The RSSI measurements are compared to the various measurementsincluded in the location fingerprint and a match is determined (e.g., ona server, such as a “cloud” server). For example, the RSSI measurementsobtained by the mobile device may be similar to the RSSI measurementsthat were obtained by the survey device at a particular reference point,and as such, the mobile device may determine that it is located at theparticular reference point. The mobile device may identify the locationthat corresponds to particular reference point (e.g., the location thatwas dropped as a pin on the map by the surveyor) and provide thatparticular location as the current location of the mobile device.Additional details about the matching process are described below. Suchmatching techniques typically employ a “probabilistic approach” in whichthe mobile device determines the reference point for which there existsthe highest probability that the mobile device is located at.

The data training phase and the positioning phase are sometimescollectively referred to as a surveying technique for determining indoorpositioning. The location fingerprint obtained by the survey device cangenerally be referred to as survey data. Such surveying techniquestypically provide an accurate estimation of the location of the mobiledevice within the venue provided the mobile device is located near areference point. However, one disadvantage of such surveying techniquesis that they require the prior surveying (e.g., data training) of avenue. If a particular portion of a venue is not surveyed (e.g., inother words, if no reference point data is obtained for locations at orproximate to a particular portion of a venue), then it may be difficultto determine the location of the mobile device when the mobile device islocated proximate to such areas, or in some cases, the locationdetermination may be relatively inaccurate. Such shortcomings may existwhen the mobile device is positioned near portions of the venue that arerestricted to the surveyor, such as private rooms and/or stores,restricted access locations, etc.

In some implementations, additional location data may be added to thefingerprint database to supplement the initial location fingerprintsurvey data. For example, harvest data (e.g., harvest traces) that areobtained in and/or around the venue may be considered for addition tothe fingerprint database. Data that is obtained by enlisting arelatively large number of people is sometimes referred to as harvestdata. A collection of data used to determine one or both of a positionand a motion of a device (e.g., over a period of time) is sometimesreferred to as “trace” data, or generally as a “trace.” Therefore, acollection of motion and/or position data obtained by enlisting arelatively large number of people may be referred to as harvest tracedata, or generally, harvest traces. Each element of harvest data can bea sample point (e.g., a location where data is sampled) including sensormeasurements obtained by the device, with the collection of samplepoints making up a harvest trace.

In general, harvest data is obtained by enlisting a relatively largenumber of people via an online medium. For example, users who run aparticular operating system and/or application on their mobile devicemay contribute harvest traces to an operator of the operating systemand/or application. The harvest data can be provided to services thatcan use the harvest data for various purposes. For example, a pluralityof users may agree to contribute harvest traces while running a mappingapplication on their mobile device. In some implementations, the usermay be required to “opt-in” before harvest traces can be contributed(e.g., to protect the privacy of the user). An operator of a differentservice or application, such as an operator of an indoor positioningsystem, may receive the harvest data from an operator of the mappingapplication and use the harvest data to improve the indoor positioningsystem, as described herein.

A harvest trace may include data that is used to identify a location ofa mobile device as well as RSSI measurements for one or more of the APsobserved during the data training phase. For example, a harvest tracemay be used to identify an ending location of the mobile device as themobile device travels from a known starting location (e.g., a locationthat corresponds to one of the surveyed reference points) to an unknownending location (e.g., a location inside a store for which survey datawas not obtained). The harvest trace may include pedestrian deadreckoning (PDR) data collected by the mobile device such as pedometermeasurements, position and/or orientation measurements obtained from agyroscope, accelerometer, and/or a compass, barometer measurements, etc.The ending location of the mobile device is identified using the PDRdata, and the ending location is correlated with the RSSI measurementfor the one or more of the APs observed when the mobile device ispositioned at the ending location. The result is a new location datapoint (e.g., a new reference point) that correlates an unsurveyedlocation to AP RSSI measurements.

The harvest trace data can be added to the fingerprint database. In someimplementations, the harvest trace data may undergo one or morefiltering stages to ensure that the data added to the fingerprintdatabase will have a positive effect (e.g., increase the accuracy ofindoor location determinations for the mobile device). The harvest tracedata may be added in a manner such that the harvest trace data has asimilar schema as the survey data collected during the data collectionphase by the survey device. The effect of adding such harvest trace datato the fingerprint database is that a radio map that represents thevenue (which is described in more detail below) can be extended. In thisway, the radio map may be updated to improve already-surveyed areasand/or extended to cover unsurveyed areas, including but not limited toindoor locations that were restricted to the surveyor and/or outdoorlocations proximate to the venue, thereby providing additional areas inand/or around the venue for which the location of a device can bedetermined with improved accuracy. Thus, when we talk about extendingthe radio map, we mean that devices located in the extended area may beable to accurately determine their respective location due to theinclusion of the harvest trace data in the form of new reference points.

FIG. 1 shows a block diagram illustrating a surveying technique 100 fordetermining positioning (e.g., indoor positioning within a venue). Thetechnique 100 includes a data training phase 110 and a positioning phase120.

During the data training phase 110, a survey device 102 (e.g., a mobilecomputing device such as a mobile phone, laptop, PDA, etc.) ispositioned at various reference points throughout the venue. The surveydevice 102 may include a user interface that is configured to display amap representation of the venue. In some implementations, a grid may beoverlaid over the map of the venue. The grid can be made up of cells(e.g., square cells, such as cell 312 of FIG. 3) having the same orsimilar dimensions. The cells may be three meters by three meters, tenmeters by ten meters, etc. The venue map may be obtained from a venuemap database. The venue map may include representations of multiplefloors of the venue, including outer boundaries of the venue, indoorobstructions (e.g., walls), etc. When the survey device 102 ispositioned at a particular reference point (e.g., within one of thecells), an operator of the survey device 102 (e.g., a surveyor) can dropa pin on the venue map indicating the particular position of thereference point that is being tested. The position may be associatedwith an (x, y) coordinate which may, in some cases, correspond tolatitude/longitude coordinates.

The surveyor can bring the survey device 102 to a first reference pointwithin the venue. The reference point is a location within the venue forwhich a plurality of measurements (e.g., Wi-Fi measurements) is to beobtained. Characteristics of the measurements can be obtained andstored. At some later time, a mobile device (e.g., other than the surveydevice 102) may obtain measurements at the reference point or at alocation proximate to the reference point. In general, and as describedin more detail below, the characteristics of the measurements obtainedby the mobile device can be compared to the characteristics of thestored measurements that were obtained by the survey device 102. If thecharacteristics are similar, the mobile device may determine that it ispositioned at the first reference point (e.g., within the same cell asthe first reference point).

The survey device 102 is positioned at various reference pointsthroughout the venue. Also positioned throughout the venue are aplurality of access points (APs) 104. The APs 104 may be radio frequency(RF) signal transmitters that allow Wi-Fi compliant devices to connectto a network, and in some cases, the APs 104 may be part of Wi-Firouters. At each reference point, the survey device 102 may connect to(e.g., transmit wireless signals between) each of a plurality of APs104. The survey device 102 measures one or more characteristics of thewireless signals received from each AP 104. For example, when the surveydevice 102 is positioned at the first reference point (e.g., x₁, y₁),the surveyor may drop a pin on the venue map displayed on the surveydevice 102 to indicate the location of the first reference point (x₁,y₁) within the first cell. The survey device 102 may be connected tofour APs 104—AP(1), AP(2), AP(3), and AP(4). Each of the APs 104 may beassociated with an identifier such as a media access control (MAC)address that the survey device 102 can use to identify the particular AP104. The survey device 102 can measure characteristics of signalsreceived from each AP 104, such as the received signal strengthindicator (RSSI). The RSSI can be measured for multiple wireless signalsreceived from each AP 104. The results are stored in a database 106. Thedatabase 106 is sometimes referred to as a fingerprint database, and thedata stored in the database 106 is sometimes referred to as survey data.

A plurality of measurements may be obtained for each AP 104. Forexample, for each AP 104, a wireless signal may be received at setintervals (e.g., every second) and the RSSI may be measured for eachwireless signal. The wireless signals may be received and the RSSI maybe measured under different conditions. For example, tens or hundreds ofmeasurements may be taken during a first period with the survey device102 in a first orientation. The orientation of the survey device 102 maybe adjusted, and additional measurements may be taken. Measurements maybe taken when the venue is occupied with a relatively large number ofpeople, when the venue is largely empty, when the venue is completelyempty, etc. Measurements may be taken when indoor obstructions, doors,etc. are in various open/closed states. Measurements may be taken underdifferent climate conditions. The measurements may be taken under such awide variety of circumstances to provide a relatively large data set forthe particular reference point that is comprehensive and includes thevariety of circumstances that may exist when a mobile devicesubsequently tries to determine its location in the positioning phase120.

Once a sufficient number of measurements are obtained for the firstreference point that is located at (x₁, y₁), an entry 108 for the firstreference point (x₁, y₁) is stored in the database 106. The entry 108(e.g., sometimes referred to as an element of survey data or an entry108 of survey data) includes the coordinates of the reference point andthe various RSSI measurements for each of the APs 104. The survey device102 can be positioned at a second reference point (x₂, y₂) at a secondcell of the grid overlaid on the map of the venue, and a similar processcan be repeated to obtain an entry 108 for the second reference point(x₂, y₂), which can likewise be stored in the database 106. Thecollection of survey data entries 108 stored in the database 106 issometimes referred to as the “location fingerprint” of the venue.

The positioning phase 120 occurs after at least some of the locationfingerprint of the venue (e.g., the entries 108 stored in the database106) has been obtained. During the positioning phase 120, a mobiledevice 112 (e.g., which is typically different than the survey device102) that is located at the venue may attempt to determine its location.In a similar fashion as that described above with respect to the datatraining phase 110, the mobile device 112 receives wireless signals fromone or more of the APs 104 positioned throughout the venue. The mobiledevice 112 can measure characteristics of the received wireless signals.For example, the mobile device 112 may obtain RSSI measurements 114 ofwireless signals received from each of the various APs 104. The RSSImeasurements 114 are compared 116 to the location fingerprint (e.g., thesurvey data) stored in the database 106, and based on the comparison, alocation 118 of the mobile device 112 is determined.

Multiple different techniques may be used for comparing 116 the locationfingerprint stored in the database 106 to the RSSI measurements 114. Insome implementations, a probabilistic approach is used. The locationfingerprint (e.g., the plurality of data included in the various surveydata entries 108) can be used to create RSSI probability distributionsof all APs 104 at all reference points.

FIG. 2 shows an example of an RSSI probability distribution graph 200that includes, for example, all RSSI measurements (e.g., which areincluded in the survey data entries 108 stored in database 106) obtainedfrom one of the APs 104 (e.g., AP(1)) at the first reference point (x₁,y₁). In other words, while FIG. 1 shows that the database 106 includes asingle RSSI measurement for AP(1) at the first reference point (x₁, y₁),which is denoted at RSSI₁ in the first entry 108, in practice, arelatively large number of RSSI measurements are typically taken andincluded in the database 106.

The various RSSI measurements taken during the data training phase 110can be used to infer a probability that a device positioned at or nearthe particular reference point (x₁, y₁) will receive a signal having aparticular RSSI value from the particular AP(1). In this example, theRSSI probability distribution graph 200 may include hundreds of RSSImeasurements that were obtained by the survey device 102 based onwireless signals received from AP(1) when the survey device 102 waspositioned at the first reference point (x₁, y₁). The number ofmeasurements taken during the data training phase 110 having the variousparticular RSSI values corresponds to the probability that a futuremeasurement taken by a device (e.g., the mobile device 112) will havethe various particular RSSI values when the device is positioned at thefirst reference point (x₁, y₁).

In this example, the RSSI probability distribution graph 200 indicatesthat a device positioned at the first reference point (x₁, y₁) shouldmost often receive a wireless signal from AP(1) that has an RSSI valueof about 60-dBm. In particular, a device positioned at the firstreference point (x₁, y₁) should receive a wireless signal from AP(1)that has an RSSI value of about 60-dBm about 22% of the time. Therefore,during the positioning phase 120, if the mobile device 112 receives awireless signal from AP(1) that has an RSSI value of about 60-dBm, thereis a reasonable probability that the mobile device 112 is located at thefirst reference point (x₁, y₁).

In practice, the probabilistic approach typically includes otherconsiderations than the brief example described above. For example, theRSSI probability distribution graph 200 shown in FIG. 2 only correspondsto a single one of the APs 104 at a single one of the reference points.In practice, the RSSI probability distributions for all APs 104 at allreference points will be determined and stored in the database 106. Whenthe position of the mobile device 112 is determined during thepositioning phase 120 by comparing 116 the location fingerprint (e.g.,expressed as RSSI probability distributions) to the RSSI measurements114, a plurality of comparisons 116 are performed to find a match (e.g.,the best match). For example, the RSSI measurement 114 that correspondsto AP(1) (e.g., RSSI₁) is compared to the RSSI probability distributionsfor AP(1) for each of the reference points, the RSSI measurement 114that corresponds to AP(2) (e.g., RSSI₂) is compared to the RSSIprobability distributions for AP(2) for each of the reference points,etc., and a collective comparison 116 is performed to determine the bestcollective match.

In an example, once a RSSI probability distribution of measurements isobtained for each of the APs 104 at each of the reference points, thedata are fit to a particular probability distribution having aparticular probability density function, such as a Rayleighdistribution. A Rayleigh distribution is characterized by theprobability density function:

${f\left( {x;\sigma} \right)} = {\frac{x}{\sigma^{2}}e^{{- x^{2}}/{({2\; \sigma^{2}})}}}$

where x is the RSSI and a is the shape parameter. Using the survey dataentries 108 obtained for each AP 104 at each of the reference point, aRayleigh distribution is created for each of the APs 104 at eachreference points. For each probability density function, the value for ais based on the RSSI measurements of the survey data entries 108obtained during the data training phase 110.

Subsequently during the positioning phase 120, when the mobile device112 is positioned at an unknown position, the RSSI measurement 114 foreach AP 104 can be entered into each probability density function forthe corresponding AP 104, where each probability density functioncorresponds to one of the reference points. For example, the RSSImeasurement 114 for AP(1) is entered into the probability densityfunction for AP(1) at reference point #1, the probability densityfunction for AP(1) at reference point #2, etc. Each probability densityfunction returns a probability expressed as a value between 0 and 1. TheRSSI measurement 114 for AP(2) is entered into the probability densityfunction for AP(2) at reference point #1, the probability densityfunction for AP(2) at reference point #2, etc. This process may berepeated for all probability density functions for all APs 104 at allreference points. In some implementations, other techniques may beemployed to minimize the number of computations that must take place.For example, reference points that are very far away from apreviously-determined location, or reference points that require arelatively long path of travel due to being located behind a lengthybarrier, may be discounted because it may be impossible for the mobiledevice 112 to travel such a large distance in the interval of timebetween location determinations.

Once all probabilities are computed, the probabilities that correspondto reference point #1 are multiplied together. For example, theprobability for the RSSI measurement 114 that corresponds to AP(1)(e.g., RSSI₁) at reference point #1 is multiplied by the probability forRSSI₂ at reference point #1, multiplied by the probability for RSSI₃ atreference point #1, etc. The probabilities that correspond to referencepoint #2, reference point #3, etc. are similarly multiplied together.The reference point that gives the highest cumulative probability isidentified as the location for which there is the highest likelihoodthat the mobile device 112 is located. Such a probabilistic approach issometimes referred to as a maximum likelihood test.

In some implementations, a weighted averaging technique may be used forthe comparison 116 to determine the best collective match. For aparticular comparison 116, each of the APs 104 may be assigned a levelof importance. The APs 104 of relatively higher importance are weightedmore heavily in the weighted average, and the APs 104 of relativelylower importance are weighted less heavily in the weighted average. Forexample, if a particular AP 104 (e.g., AP(5)) is assigned the highestlevel of importance, and the RSSI measurement 114 that corresponds toAP(5) (e.g., RSSI₅) provides a relatively high probability value for theprobability density function for AP(5) at a particular reference point,then there may be a high likelihood that the particular reference pointis chosen as the best match. On the other hand, if a particular AP 104(e.g., AP(8)) is assigned the lowest level of importance, then even ifthe RSSI measurement 114 that corresponds to AP(8) (e.g., RSSI₈)provides a relatively high probability value for the probability densityfunction for AP(8) at a particular reference point, the match may have aminimal effect on the comparison decision, and there may be a lowlikelihood that the particular reference point is chosen as the bestmatch.

In some implementations, the level of importance used in the weightedaverage may be based at least in part on the magnitude of the RSSImeasurements 114 that correspond to the various APs 104. For example, itmay be inferred that stronger RSSI measurements 114 are more accuratebecause the user is likely closer to those corresponding APs 104.Therefore, the APs 104 that correspond to the stronger RSSI measurements114 may be more heavily weighted in the weighted average.

In some implementations, the venue and areas in proximity to the venuemay be expressed as a graphical map, sometimes referred to as a radiomap. The radio map is associated with the survey data obtained duringthe data training phase 110, as well as other location data, asdescribed in more detail below. The term radio map originates from theassociation of the graphical map with such location data that is basedon characteristics of radio signals (e.g., Wi-Fi signals). FIG. 3 showsan example of a radio map 300 for a venue (e.g., a mall) that includesat least four stores, Store A-D. The surveying technique described abovewith respect to FIG. 1 may be employed in the mall.

During the data training phase 110, a surveyor may bring a survey device(e.g., the survey device 102 of FIG. 1) to each of a plurality ofreference points 302, represented as black triangles in theillustration. The reference points 302 may be located in generallyaccessible areas of the venue, such as hallways, concourses, lobbies,etc. In some implementations, a grid may be overlaid over the map of themall. The grid can be made up of cells. In some implementations, some orall of the cells are square cells having the same or similar dimensions(e.g., between three meters by three meters and ten meters by tenmeters, although smaller or larger dimensions can also be used). In someimplementations, the grid may be made up of cells of various shapes andsized.

The cells have the effect of binning data obtained by the survey device102. The cells can also be used as a visual aid for the surveyor toindicate locations for which survey data is to be obtained. For example,the survey device 102 may include a user interface that is configured todisplay the radio map 300 (or, e.g., a modified version of the radiomap) with the overlaid grid. Once the surveyor is positioned at aparticular reference point 302 (e.g., within one of the cells), he orshe may provide an input through the user interface (e.g., a touchinput) to indicate the location of the reference point 302 to be tested.For example, the surveyor may drag and drop a pin into the correspondingcell on to the radio map 300 to indicate the particular reference point302 at which the survey device 102 is currently positioned.

A plurality of APs 304 (e.g., such as the APs 104 of FIG. 1) may bedistributed throughout the mall. The APs 304 may be positioned inhallways/corridors of the mall, in the stores, outside of the mall, etc.Once the survey device 102 is positioned at the particular referencepoint 302 to be tested, the survey device 102 may obtain a plurality ofmeasurements from the various APs 304. For example, the survey device102 may perform a scan to determine which APs 304 the survey device 102is in wireless communication with. If the survey device 102 receives oneor more signals from a particular AP 304, the survey device 102 canrecord an identifier for the AP 304 (e.g., such as a MAC address) andalso take measurements of a characteristic of the signal (e.g., such asan RSSI measurement). The data can be stored in a database (e.g., 106 ofFIG. 1), the surveyor can bring the survey device 102 to the nextreference point 302, and the process can be repeated until data for eachdesired reference point 302 is obtained.

In some implementations, the surveyor may follow a predetermined pathand obtain data for reference points 302 at a particular distanceinterval (e.g., every three meters, every ten meters, etc.). Forexample, the surveyor may obtain data for a first reference point 302when the surveyor first enters the mall. The surveyor may then beginwalking down a hallway and obtain data for a second reference point 302after walking approximately three meters. Data for reference points 302can continue to be obtained in this fashion as the surveyor walks alongvarious paths within Building A, including traveling to different floorswithin the building. In some implementations, the surveyor may gatherdata for a number of reference points 302 such that sufficient coverageof the venue is obtained. In general, the more reference points 302 forwhich data is obtained within a venue, the more accurate the subsequentpositioning phase (e.g., 120 of FIG. 1) can be.

In some implementations, surveying may not be available for portions ofthe mall. For example, particular stores (e.g., Stores A-D) may notallow surveyors to survey within the stores. This is shown in FIG. 3 bythe absence of any reference points 302 within the stores. Because noreference points exist within the stores, the mobile device 112 thatperforms the positioning phase 120 while the mobile device 112 is insideone of the stores may be unable to accurately determine its position.For example, because no reference points exist within the stores, theRSSI measurements 116 may not closely match any of the survey dataobtained during the data training phase 110, or the RSSI measurements116 may provide a poor match that results in the positioning phase 120determining a location for the mobile device 112 that does not match itstrue location. For example, the mobile device 112 may be inside Store Aat the upper wall, but the positioning phase 120 may determine that themobile device 112 is at the reference point 304 near the entrance ofStore A. Therefore, to improve the ability of the positioning phase 120to determine positions of mobile devices 112 located at unsurveyedareas, additional reference points can be added to such unsurveyedareas. Adding such additional reference points is sometimes referred toas extending the radio map 300 (e.g., extending one or more borders ofthe radio map 300 to form an extended radio map).

In the illustrated example, the radio map 300 may be extended to coverareas for which survey data was not obtained. For example, the radio map300 is extended by including a new reference point (e.g., a referencepoint that was not included in the initial version of the radio map 300.Such new reference points are referred to herein as extended referencepoints. In the illustrated example, the radio map 300 is extended intoStore A by including an extended reference point P₁ 310, identified as ablack circle. The extended reference point P₁ 310 is different than thereference points 302 identified by black triangles in that the extendedreference point P₁ 310 was not obtained by the data training phase 110of the surveying technique 100. Rather, extended reference point P₁ 310is obtained by taking different location information (e.g., other thandropping a pin on a map, as described in more detail below). However,once the extended reference point P₁ 310 is obtained and added to theradio map 300, thereby extending the radio map 300, the extendedreference point P₁ 310 may be treated by the radio map 300 and thepositioning phase 120 the same way that the surveyed reference points302 are treated. In other words, from the perspective of the radio map300 and the positioning phase 120, the extended reference point P₁ 310is simply another location that can be used to identify the currentlocation of the mobile device 112 during the positioning phase 120.

The extended reference point P₁ 310 is obtained based on harvest data306 (e.g., harvest traces). The harvest data 306 are represented asblack x's in the illustration. Each element of harvest data 306 can be asample point including, among other things, one or more sensormeasurements obtained by a device (e.g., a mobile device). The harvestdata 306 shown in FIG. 3 make up a trace (e.g., a harvest trace). Thatis, a trace is a collection of sample points of harvest data 306. Thetrace may be obtained as the device travels along a particular path.Using the harvest data 306 for a particular trace, in some cases byemploying a regression technique such as a least squares technique usinga Kalman filter (e.g., a forward-backward Kalman filter), a trajectory308 is determined. In some implementations, the trajectory 308 isoptimized to improve its accuracy, as described in more detail below.The trajectory 308 is a determination of a motion path traveled by thedevice. Based on the trajectory 308, various locations of the deviceover time can be determined.

In some implementations, each element of harvest data 306 includes i)data used to identify a location of the device, and ii) RSSImeasurements for one or more of the APs 304. The data used to identifythe location of the device (e.g., sometimes generally referred to assensor data, motion data, dead reckoning data, etc.) includesmeasurements obtained by sensors of the device (e.g., one or moregyroscopes, accelerometers, compasses, barometers, etc. The elements ofharvest data 306 may be obtained at a frequency of approximately 1 Hz(e.g., one element of harvest data 306 per second), and may include aspeed and a heading rate. In some implementations, one or both of thespeed and the heading rate (or, e.g., the sensor measurements used todetermine the speed and heading rate) may be downsampled (e.g., at afrequency of less than 1 Hz). Such downsampling may be performed toreduce the resolution of the data in order to protect the privacy of theuser.

The speed may be determined based on measurements obtained by apedometer, accelerometer, and/or gyroscope. For example, a step countmay be obtained by the pedometer, and a stride length (e.g., a distancetraveled per step) may be determined based on measurements obtained bythe accelerometer and/or the gyroscope. Based on the step count and thestride length, a distance traveled by the device can be indirectlydetermined. Thus, using the step count and the stride length over anelapsed time, the speed of the device can be determined. The headingrate may be determined based on measurements obtained by a compass, thegyroscope, the accelerometer, a magnetometer, etc. In particular, theheading rate may be derived based on a change of attitude of the deviceas measured by one or more of the compass, the gyroscope, theaccelerometer, the magnetometer, etc. The heading rate can be integratedby the device to determine a heading. Therefore, for each element ofharvest data 306, a value for the speed and a value for the heading ofthe device is determined.

Each element of harvest data 306 also includes RSSI measurements for oneor more of the APs 304. Therefore, once the trajectory 308 is determinedbased on the harvest data 306, one or more locations on or proximate tothe trajectory 308 (e.g., such as extended reference point P₁ 310) areidentified as described below, and such locations can be correlated tothe RSSI measurements to create additional location fingerprint data ina similar fashion as described above with respect to the locationfingerprint survey data. The RSSI measurements for each AP 304 at eachextended reference point may be represented as RSSI probabilitydistributions in a manner similar to that described above with respectto FIG. 2. Probability density functions may be obtained (e.g., in theform of Rayleigh distributions), and the probability density functionsmay be used for determining the location of the mobile device 112 insubsequent positioning phases 120 using a maximum likelihood test, asdescribed above.

In some implementations, the harvest data 306 may be filtered before itis added to the existing location fingerprint survey data that is storedin the database 106. Thereafter, during the positioning phase 120, thelocation of the device when the device is positioned at or nearunsurveyed areas (e.g., such as within Store A) may be determined. Whenwe talk about extending the radio map 300, we mean that devices locatedin the extended area (e.g., within Store A) may be able to accuratelydetermine their respective location due to the inclusion of theadditional location data in the form of extended reference points.

FIG. 4 is a block diagram of an exemplary process 400 of extending theradio map 300 using harvest traces (e.g., based on the harvest data306). A plurality of traces (e.g., Trace 1 402 a, Trace 2 402 b, Trace n402 n, etc.) are obtained from a plurality of devices. In someimplementations, each trace may correspond to harvest data 306 obtainedby a single device over a particular period of time. For example,referring again to FIG. 3, a single trace is illustrated which includesall of the illustrated harvest data 306. Additional traces may beobtained by the same device (e.g., at different times, at differentlocations, etc.) or by other devices contributing to the harvest data.Each trace is provided to a localizer 404 that is configured todetermine an optimized trajectory based on the harvest data 306 for theparticular trace. The optimized trajectories, which are alsosupplemented with the RSSI measurements for one or more of the APs 304,are provided to a map builder 406.

Before receiving the optimized trajectories and the corresponding RSSImeasurements, the map builder 406 builds the radio map 300 using thesurvey data entries 108 included in the fingerprint database 106 in themanner described above. In some implementations, the radio map 300 maybe received in another way, as described in more detail below. Once theoptimized trajectories and the corresponding RSSI measurements arereceived by the map builder 406, the map builder 406 can refine theradio map 300 to include additional reference points (e.g., extendedreference points, such as the extended reference point P₁ 310 of FIG.3), thereby extending the radio map 300 into unsurveyed areas.

Referring again to FIG. 3, a single trajectory 308 is illustrated inStore A. In practice, the map builder 406 may consider a plurality oftrajectories (e.g., tens, hundreds, thousands, etc.) inside Store A. Theplurality of trajectories may be collectively considered to determineappropriate locations for including as extended reference points. Forexample, extending the radio map 300 may include identifying a pluralityof trajectories (e.g., optimized trajectories) and identifying locationsin the venue (e.g., particular cells of the radio map 300) thatcorrespond to locations at or proximate to the plurality oftrajectories. In some implementations, if a threshold number of tracespass through a particular cell of the venue, the location thatcorresponds to the particular cell can be added to the radio map 300 asan extended reference point.

In the illustrated example, the trajectory 308 corresponds to locationsinside Store A. As such, the radio map 300 can be extended to include afootprint of Store A. The footprint of Store A may then be divided intoa plurality of cells. For example, a grid may be applied to thelocations at or proximate to the plurality of trajectories that includesa plurality of cells (e.g., including the cell 312). The cells may havethe same or similar dimensions as the cells of the corridor. In someexample, the cells have dimensions of between three meters by threemeters and ten meters by ten meters, although other dimensions can beused. If at least one trajectory 308 passes through a cell of the radiomap 300, the location that corresponds to the cell may be added as anextended reference point (e.g., the cell can be added to the radio map300). In this way, multiple extended reference points can be added tothe radio map 300 based on location information included in a singletrajectory.

In some implementations, a location at or near a particular trajectory(or, e.g., a plurality of trajectories) may be determined to beappropriate for addition to the radio map 300 as an extended referencepoint based on one or more factors. In some implementations, an amountof harvest data 306 that is available for a particular location mayfactor into the determination of whether the particular location is tobe included as an extended reference point. For example, a location maybe included as an extended reference point if a particular number ofelements of harvest data 306 (e.g., from one or more trajectories) thatcorrespond to the cell of the radio map 300 are available. In someimplementations, a location may be included as an extended referencepoint if a threshold number of trajectories that pass through thecorresponding cell of the radio map 300 are available. In someimplementations, one or more indicators of the quality of the harvestdata 306 may be considered in determining whether the correspondinglocation are to be added to the radio map 300. For example, if muchharvest data 306 for a particular location is available, but the qualityof the harvest data 306 is below a quality threshold, the location maynot be added as an extended reference point. In contrast, if harvestdata 306 is determined to be of high quality (e.g., meeting a qualitythreshold), the corresponding location may be added as an extendedreference point even if a relatively small quantity of harvest data 306is available for the particular location. In some implementations, thequality of the harvest data 306 may be determined based at least in parton a horizontal accuracy of the elements of harvest data 306. In someimplementations, the quality of the harvest data 306 may be determinedat least in part based on the calculations performed by the localizer(404 of FIG. 4) when providing an optimized trajectory, as described inmore detail below.

In some implementations, any location that resides at or is proximate toa trajectory may be considered for including as extended referencepoints. In other words, any cell of the venue through which a trajectorypasses may be added to the radio map 300. In this way, any location thatresides on a trajectory may be an appropriate location for including asan extended reference point.

Applying the grid may have the effect of binning the harvest data 306.For example, five elements of harvest data 306 reside within the cell312, so those five elements of harvest data 306 are determined tocorrespond to a particular location (e.g., a single location). In someexamples, the particular location has coordinates that correspond to thecenter of the cell 312. As such, the particular location at the centerof the cell 312 is identified as being the extended reference point P₁310. The RSSI measurements that correspond to the five elements ofharvest data 306 are correlated with the extended reference point P₁310. The RSSI measurements may be used to form a probability densityfunction that corresponds to the extended reference point P₁ 310. Insome implementations, rather than the extended reference points beingassigned to the center of the cell 312, an averaging technique and/or aclustering technique may be applied to the plurality of trajectories toidentify suitable locations to be used as extended reference points.

Additional extended reference points may be created for the other cellsthat the trajectories pass through. In the illustrated example, extendedreference points may be created for the other nine cells that thetrajectory 308 passes through along the perimeter of Store A. However,the trajectory 308 provides insufficient data for identifying extendedreference points in the inner two cells of Store A. Other trajectories(e.g., based on other traces such as Trace 1 402 a, Trace 2 402 b, Tracen 402 n, etc.) may subsequently be used to identify additional extendedreference points and further extend the radio map 300.

Even after the radio map 300 is generated by the map builder 406 usingboth the survey data of the fingerprint database 106 and the extendedreference points (e.g., based on the optimized trajectories provided bythe localizers 404), the radio map 300 can be constantly updated (e.g.,extended) and/or optimized in an iterative manner. For example, thearrow emanating from the left side of the radio map 300 block andtraveling back to the localizers 404 indicates that the radio map 300may constantly consider new harvest data and computed optimizedtrajectories to extend into additional unsurveyed areas. In this way,the radio map 300 can be used in a simultaneous localization and mapping(SLAM) manner in which the radio map 300 can provide the location of adevice while simultaneously constructing and/or updating the radio map300.

In some implementations, the radio map 300 may be progressively extendedby applying the same harvest data through a plurality of passes. In thisway, a confidence in the corresponding extended reference points can beincreased. Such an approach may be especially beneficial for extendedreference points that are located relatively deeper in unsurveyed areas,which may have a tendency to include increased compounded errors ascompared to extended reference points that are located in relativelymore shallow locations in unsurveyed areas. In some implementations,such a progressive extension approach may require a relatively largeamount of uncorrelated traces over the same extended map areas (e.g., toprevent the iteration from leading to biases). In some implementations,one or more techniques may be employed on the traces during themap-building process, such as a leave-one-out cross-validationtechnique, to minimize possible biases.

While the localizers 404 as shown as having the same reference number(e.g., indicating that they are the same component), in someimplementations, the localizer used from one trace (e.g. Trace 1 402 a)to another (e.g., Trace 2 402 b) may be different.

FIG. 5 is a block diagram of an exemplary localizer (e.g., the localizer404 of FIG. 4) that accepts a trace (e.g., Trace n 402 n) as input andprovides an optimized trajectory 516, which can then be provided to themap builder 406 to extend the radio map 300.

Referring to FIGS. 3 and 5 together, the trace 402 n may be made up fromthe elements of harvest data 306 within and in proximity to Store A. Forexample, a user (e.g., a user contributing to the harvest data 306)carrying a device may enter Store A from a mall corridor. The userenters Store A from a location in the mall for which survey data wasobtained. For example, a reference point 302 exists at the doorway intoStore A. Therefore, as the user enters Store A and upon exiting Store A,the location of the user can be determined to a relatively high degreeof accuracy. Once the user enters Store A, there may not be survey dataavailable, and there may not be any additional (e.g., extended)reference points available. However, upon entering and travelingthroughout Store A, the harvest data 306 are being collected by theuser's device, and in some cases, being provided to a server (e.g., a“cloud” server).

At a given frequency (e.g., a frequency of about 1 Hz), elements ofharvest data 306 are collected by the user's device. Each element ofharvest data 306 includes data that can be used to identify a locationof the device. Such data is referred to as dead reckoning data 502. Eachelement of dead reckoning data 502 includes a speed and a heading rate.The speed may be determined based on measurements obtained by apedometer, accelerometer, and/or gyroscope of the device. For example, astep count may be obtained by the pedometer, and a stride length may bedetermined based on measurements obtained by the accelerometer and/orthe gyroscope. Based on the step count, the stride length, and anelapsed time, the speed of the device can be determined. The headingrate may be determined based on measurements obtained by a compass, thegyroscope, the accelerometer, a magnetometer, etc. The heading rate canbe integrated by the device to determine a heading. Therefore, for eachelement of dead reckoning data, a value for the speed and a value forthe heading of the device is determined.

While survey data is typically accurate and reliable because thelocation that corresponds to each reference point was manually input bya human user, the dead reckoning data 502 may include inherentinaccuracies. For example, while the speed and heading rate may be knownat one-second intervals, and while theoretically the location of thedevice may be determined based on such information, such errors tend toaccumulate as the user travels throughout the store. For example, theuser may be traveling in a straight line, but the dead reckoning data502 may indicate that the user is “drifting” (e.g., departing from astraight line path). Such errors may compound until the location of theuser can be known with a relatively high degree of certainly. Forexample, when the user exits Store A and is again in proximity tosurveyed reference points 302, the location of the user is known, andsuch reliable information can be considered when computing the user'strajectory. Such locations that are determined with a high degree ofcertainty are sometimes referred to as anchors. Using anchors, acomputed trajectory that is drifting can be corrected with more-reliablelocation information.

Using the user's speed and heading rate as inputs, a four-dimensionaldynamics model can be used to determine the user's change in position asfollows:

$\begin{bmatrix}\overset{.}{x} \\\overset{.}{y} \\{\overset{.}{q}}_{1} \\{\overset{.}{q}}_{2}\end{bmatrix} = {{\begin{bmatrix}0 & 0 & \upsilon & 0 \\0 & 0 & 0 & \upsilon \\0 & 0 & 0 & {- \omega} \\0 & 0 & \omega & 0\end{bmatrix}\begin{bmatrix}x \\y \\q_{1} \\q_{2}\end{bmatrix}} = {\begin{bmatrix}q_{1} & 0 \\q_{2} & 0 \\0 & {- q_{2}} \\0 & q_{1}\end{bmatrix}\begin{bmatrix}\upsilon \\\omega\end{bmatrix}}}$

where (x, y) is the user's position, v is the user's speed, q₁=cos θ,q₂=sin θ, θ is the user's heading, and ω is the user's heading rate.

A trajectory for the trace 402 n can be determined based on the variouscomputed changes in position using a least squares optimization (e.g.,least squares stage 1 504) according to the following function:

${J\mspace{11mu} \left( \; {{.\;.\;.}\mspace{14mu},{\overset{{x\; 0},\; {y\; 0},\; {{theta}\; 0}}{{\hat{v}}_{k},}\;.\;.\;.}\mspace{14mu},{\hat{\omega}}_{k},\; {.\;.\;.}}\mspace{11mu} \right)}\; = {{\sum\limits_{k = 1}^{N_{d}}\left\lbrack {{\sigma_{v}^{- 2}\left( {{\overset{\_}{v}}_{k} - {\hat{v}}_{k}} \right)}^{2} + {\sigma_{\omega}^{- 2}\left( {{\overset{\_}{\omega}}_{k} - {\hat{\omega}}_{k}} \right)}^{2}} \right\rbrack} + {\sigma_{p}^{- 2}{\sum\limits_{k = 1}^{N_{p}}\left\lbrack {\left( {{\overset{\_}{x}}_{k} - \hat{x_{k}}} \right)^{2} + \left( {{\overset{\_}{y}}_{k} - {\hat{y}}_{k}} \right)^{2}} \right\rbrack}}}$

where v _(k) and ω _(k) are the measured dead reckoning data 502 asinputs, {circumflex over (v)}_(k) and {circumflex over (ω)}_(k) are theestimated dead reckoning data, x _(k) and y _(k) are the measureduser/device position, and {circumflex over (x)}_(k) and ŷ_(k) are theestimated user/device position (e.g., derived from the estimated deadreckoning data). Based on the least squares optimization function, anestimated trajectory is formed. The estimated trajectory is optimizedaccording to one or more processes, as described in detail below, andthe eventual result is an optimized trajectory 516.

Given an initial state (e.g., given by x₀, y₀, θ₀) in the least squaresoptimization function, the dynamics model can be used to propagate theinitial state combined with the input (e.g., {circumflex over (v)}_(k)and {circumflex over (ω)}_(k)) to obtain the position of the user/deviceat any point in time. The least squares optimization function weighs theposition and the input at each time k by taking a difference between themeasured positions (e.g., x _(k) and y _(k)) and the estimated positions(e.g., {circumflex over (x)}_(k) and ŷ_(k)) and a difference between themeasured dead reckoning data (e.g., v _(k) and ω _(k)) and the estimateddead reckoning data (e.g., {circumflex over (v)}_(k) and {circumflexover (ω)}). Relatively small differences between the measured data andthe estimated data indicate accurate predictions. The time k correspondsto the interval at which each element of harvest data 306 is obtained(e.g., once per second). In some implementations, the least squaresoptimization function may include additional terms. In someimplementations, additional terms may be provided at a separate leastsquares stage (e.g., the least squares stage 2 514).

The Trace n 402 n also includes RSSI measurements 506 for at least someof the elements of harvest data 306. For example, for a given element ofharvest data 306, which corresponds to an estimated location on atrajectory determined according to the least squares optimizationfunction above, RSSI measurements 506 obtained by the device when thedevice was at the estimated location on the trajectory are also known.The RSSI measurements 506 and the radio map 300 are provided to a Wi-Fioptimizer 508. The functionality of the Wi-Fi optimizer 508 may dependon the current state of the radio map 300. For example, if the radio map300 currently only includes surveyed reference points 302, then the RSSImeasurements 506 may only be useful if they are obtained from locationsat or proximate to such surveyed reference points 302 (e.g., locationsnear the entrance of Store A). For example, suppose the least squaresoptimization function employed at the least squares stage 1 504indicates that the user traveled into Store A, walked around insideStore A, and walked through the right wall of Store A into Store B. Uponthe user in fact exiting Store A, the Wi-Fi optimizer 508 can determinea Wi-Fi position 510 of the user based on the RSSI measurements 506 andthe existing radio map 300 including the surveyed reference points 302.Because survey data is relatively reliable, the reference point 302located near the entrance of Store A is identified as being the locationof the device despite the least squares optimization functionidentifying the location as being somewhere in Store B. In thisimplementation, the reference point 302 located near the entrance ofStore A acts as an anchor. In this way, the Wi-Fi position 510determined by the Wi-Fi optimizer 508 can provide input into the leastsquares stage 1 504 to provide a better estimate for the user'strajectory.

Now suppose the radio map 300 includes extended reference points (e.g.,the extended reference point P₁ 310) that were obtained previously bythe technique generally described herein. Such an extended referencepoint P₁ 310 may not be quite as reliable as surveyed reference points302, but may still be relatively accurate, especially after somerefinement by the positioning system. Such an extended reference pointP₁ 310 may also be used as an anchor point for the dead reckoning data502. In other words, if the dead reckoning data 502 causes the leastsquares optimization function to provide a given trajectory thatincludes drift errors, the Wi-Fi optimizer 508 can use the RSSImeasurements 506 taken by the device, and the radio map 300 thatincludes the extended reference point P₁ 310, to determine a Wi-Fiposition 510 that can be considered by the least squares stage 1 toassist in correcting the drift error. When the RSSI measurements 506 ofTrace n 402 n indicate that a close match has been obtained for theprobability density function that corresponds to the extended referencepoint P₁ 310, the trajectory can be anchored to the location of theextended reference point P₁ 310 at the corresponding time, and the deadreckoning data 502 can be essentially reset such that any driftexperienced up to that point is no longer causing a cumulative effect inthe drift error. While drift error may still exist in the dead reckoningdata 502 as the user travels in a clockwise direction around and out ofStore A, such drift error will be minimal compared to the amount ofdrift error that would exist if no anchoring occurred inside Store A. Asadditional extended reference points are added to the radio map 300, andas existing extended reference points become refined by the employedSLAM technique, the location determination capability of the systeminside unsurveyed areas is continuously expanded and improved.

In some implementations, the localizer 404 may also include a leastsquares stage 2 that considers input from an occupancy map 512. Theoccupancy map 512 is a representation of data that may be availableand/or incorporated in a radio map 300. The occupancy map 512 indicateslocations within the venue that cannot be occupied by users. Forexample, the occupancy map 512 may indicate that particular cells of theradio map 300 cannot be occupied because it is impossible for a user tooccupy them (e.g., the location is inside a wall) or because suchlocations are restricted (e.g., private rooms inaccessible to thegeneral public). Thus, if a position of a user is identified as being ata location that cannot be occupied, a decision can be made that thedetermined location is incorrect.

In some implementations, the occupancy map 512 and related informationis provided to the least squares stage 2 514. In some implementations,the least squares stage 2 514 may simply be included in the form of anadditional term to the least squares stage 1 504. If the estimatedposition (e.g., {circumflex over (x)}_(k) and ŷ_(k)) is identified as alocation that can be occupied (e.g., a walkable location) according tothe occupancy map 512, then zero cost may be contributed to theleast-squares optimization at the least squares stage 2 514. If theestimated position is identified as a location that cannot be occupied(e.g., non-walkable), then a quadratic cost (e.g., an error componentthat increases exponentially based on a quantity, in this case adistance) may be contributed to the least-squares optimization at theleast squares stage 2 514. The quadratic cost may be relatively greaterthe further away the estimated position is from a walkable location. Inother words, if the estimated position is determined to be at a locationthat is non-walkable, it can be de-weighted according to a distancebetween the estimated position and the closest walkable locationprovided by the occupancy map 512.

Following the least squares stage 2, the optimized trajectory 516 isprovided to the map builder 406. The trajectory 516 is optimized in thesense that dead reckoning data 502 is initially used to obtain a generaltrajectory, but due to known inherent errors in the dead reckoning data502, other techniques are applied to the general trajectory to minimizesuch errors and obtain an optimized trajectory 516 that is a moreaccurate representation of the actual path traveled by the user.

In some implementations, the localizer 404 may include one or moreadditional algorithms to assist in providing the optimized trajectory516. For example, in some implementations, the localizer 404 may includea dead reckoning particle filter.

The localizer 404 can operate on the server or on the user's device todetermine the optimized trajectory 516 and determine an estimatedlocation of the user's device. In this way, the user can utilize thedead reckoning data 502 as well as the radio map 300 and the Wi-Fioptimizer 508 to determine a current location. The optimized trajectory516 can also be provided to the map builder 406 to extend the radio map,as described above with respect to FIG. 4. For example, a similarprocess can be performed for other traces made up of other harvest data306 (e.g., for a plurality of users, using a plurality of devices, atvarious times, traveling at various locations within the mall, etc.) toobtain a plurality of optimized trajectories. The map builder 406 canconsider the plurality of optimized trajectories, identify particularlocations in the optimized trajectories as extended reference points,and correlate such extended reference points with RSSI measurements tocontinuously extend the radio map 300 to cover additional unsurveyedlocations. Therefore, users who subsequently use the indoor positioningsystem will have additional reference points available to them toimprove the location determination decision.

The representation of the localizer 404 illustrated in FIG. 5 includeselements that may or may not actually be part of the localizer 404, suchas the input Trace n 402 n including the dead reckoning data 502 and theRSSI measurements 506, the input radio map 300, and/or the outputoptimized trajectory 516. Such elements are displayed as part of thelocalizer 404 block diagram for ease of viewing.

FIG. 6 is a flowchart of an exemplary process 600 of extending a radiomap (e.g., the radio map 300 of FIG. 3). The process 600 can beperformed, for example, by the electronic device (e.g., a server)described with respect to FIG. 7, or the computing device (e.g., amobile computing device) described with respect to FIG. 8. At step 602,a radio map (e.g., an indoor radio map) is built using the survey data.The radio map includes at least one boundary. Referring to the radio map300 of the mall as an example, the initial radio map 300 includes aboundary at the bottom walls of Store A and Store B and a boundary atthe top wall of Store C and Store D. In other words, because the initialradio map 300 is built using survey data, and thus includes surveyedreference points 302 in the corridor of the mall but no extendedreference points (e.g., the extended reference point P₁ 312), theinitial radio map 300 is bound at least by the walls between thecorridor and Stores A-D.

At step 604, harvest data (e.g., the harvest data 306 of FIG. 3) isreceived from a mobile device. As described above, the harvest data 306may be harvest traces, where the collection of harvest data 306 make upa harvest trace. Each element of harvest data 306 can be a sample pointincluding, among other things, one or more sensor measurements obtainedby the mobile device (e.g., used to identify a location of the mobiledevice) and RSSI measurements for one or more of the APs 304 in orproximate to the mall. The harvest data 306 may be obtained by themobile device while a user carries the mobile device across variouslocation inside and outside the boundary of the radio map 300. Forexample, a user may carry a mobile device in his pocket as he walksalong the corridor of the mall. As the user walks, the harvest data 306may be obtained at a rate of approximately 1 Hz. Harvest data 306 may beobtained while the mobile device is positioned inside the boundary ofthe initial radio map 300 (e.g., outside of the entrance to Store A) aswell as while the mobile device travels outside the boundary of theinitial radio map 300 (e.g., inside of Store A).

At step 606, based on the harvest data 306, a trajectory (e.g., thetrajectory 308) of the mobile device is determined. In particular, theone or more sensor measurements obtained by the mobile device are usedto determine (e.g., continuously or substantially continuously) alocation of the mobile device as the mobile device travels within StoreA. For example, because no surveyed reference points 302 exist withinStore A, the positioning phase 120 cannot be reliably used to determinethe location of the mobile device while the mobile device is withinStore A. The one or more sensor measurements include measurementsobtained by a pedometer, accelerometer, gyroscope, compass,magnetometer, etc. of the mobile device. The one or more measurementsare used to determine a speed and a heading rate of the mobile device asthe mobile device travels within Store A. Using the technique describedabove with respect to FIGS. 4 and 5, the trajectory 308 of the mobiledevice is determined based on the computed speed and heading rate. Atleast some of the trajectory 308 resides outside of the initial boundaryof the radio map 300. In other words, a substantial portion of thetrajectory 308 resides inside of Store A (e.g., because the trajectory308 is used to identify locations for which survey data does not exist).

At step 608, one or more locations on or proximate to the trajectory areidentified. In the illustrated example, one of the identified locationsis the extended reference point P₁ 310. The extended reference point P₁resides both at the center of the cell 312 and directly on thetrajectory 308. However, in some examples, the trajectory 308 may notpass through the center of the cell, yet the center of the cell may beset as the identified location. For example, referring to the cell 314,the trajectory 308 does not pass through the center of the cell 314, yetdue to the binning technique employed, the center of the cell 314 may beset as the identified location, and the harvest data 306 that resideswithin the cell 314 may be determined to correspond to the identifiedlocation at the center of the cell 314.

The extended reference point P₁ 310 can be used as an additionalreference point to be used during a subsequent positioning phase 120 ina manner similar to the reference points 302 obtained by the surveydevice 102. For example, in addition to including sensor measurements(e.g., dead reckoning data) for determining a position of the mobiledevice, each element of harvest data 306 also includes RSSI measurementsfor one or more of the APs 304 in or proximate to the mall with whichthe mobile device is in communication. Thus, the position of theextended reference point P₁ 310, which resides in the cell 312, can becorrelated with a set of RSSI measurements that correspond to, forexample, the five elements of harvest data 306 that reside in the cell312. In practice, many additional elements of harvest data 306 may existin the cell 312 (e.g., from other harvest traces from other mobiledevices, from other harvest traces from the same mobile device, etc.).The RSSI measurements that correspond to the harvest data 306 within thecell 312 can be represented as RSSI probability distributions for eachof the APs 304 in a manner similar to that described above with respectto FIG. 2. Probability density functions may be obtained (e.g., in theform of Rayleigh distributions), and the probability density functionsmay be used for determining the location of the mobile device 112 insubsequent positioning phases 120 using a maximum likelihood test, asdescribed above. In other words, once the extended reference point P₁310 is correlated with probability density functions for each of the AP304, mobile devices 112 that are positioned inside Store A at or nearthe extended reference point P₁ 310 will be able to determine theirpositions to be at the cell 312 upon receiving RSSI measurements 114that satisfy the maximum likelihood test.

At step 610, an extended radio map is built using both the survey dataand the one or more identified locations. For example, once the one ormore identified locations are correlated with RSSI measurements (e.g.,in the form of probability density functions), the one or moreidentified locations and the corresponding RSSI measurements can be used(e.g., by the map builder 406 of FIG. 4) to extend the radio map (e.g.,build an updated version of the radio map 300). The extended radio mapis defined at least in part by an extension of the boundary. Theextended boundary encompasses the one or more identified locations on orproximate to the trajectory. In the illustrated example of FIG. 3, theboundary of the radio map 300 that is formed between the corridor andStore A is extended to encompass the perimeter of Store A. The extensionof the boundary is possible due to the inclusion of location and RSSIinformation that corresponds to the extended reference point P₁ 310.Therefore, the extended radio map now includes the cell 312 and theextended reference point P₁ included therein. In some implementations,the identified one or more locations and the corresponding RSSImeasurements are stored in the fingerprint database 106 in a similarform as the survey data entries 108. In this way, during a subsequentpositioning phase 120, the extended reference points may beindistinguishable from the surveyed reference points 302.

In some cases, harvest data may be unsuitable for supplementing thesurvey data for a number of reasons. If unsuitable harvest data is usedby the map builder 406, the net effect may be to reduce the overallaccuracy of the location determination system. Therefore, in someimplementations, the harvest data (and, e.g., the resulting optimizedtrajectories) may be examined and filtered prior to being used to extendthe radio map to ensure that the data will allow for extension of theradio map without negatively impacting the accuracy of the system. Ifthe system determines that the optimized trajectories are unreliable(e.g., there is relatively little confidence that the locations on theoptimized trajectories match the true location of the device), suchoptimized trajectories may not be considered for adding additionalextended reference points to the radio map.

In some implementations, the harvest data 306 may include one or moreindicators of the accuracy of the data. For example, the harvest datamay include a parameter for indicating that the measurements obtained byone or more of the pedometer, the gyroscope, the accelerometer, themagnetometer, etc. are particularly noisy or particularly inaccurate fora variety of reasons. Such inaccuracies may result in an inaccuratecalculation of the user's speed and heading rate, and in turn, aninaccurate computed optimized trajectory. Thus, if the parametersatisfies a predetermined threshold, the optimized trajectory computedbased on the harvest data 306 may be ignored by the map builder 406, orthe optimized trajectory may be assigned a relatively lesser weight thanother optimized trajectories that do not include such indicators of lowaccuracy.

While we have largely described the additional location data forextending the radio map as being harvest data that makes up harvesttraces, other types of location data can also or alternatively be usedto extend the radio map. For example, in some implementations, theadditional location data may be harvested GPS data that identifies a GPSlocation. For example, when a user is at a location for which GPS datais available, the user's device may determine the GPS location of thedevice as well as RSSI measurements of various APs that the device is incommunication with. Like the harvest data, the GPS data can also befiltered before being used to identify additional reference points forextending the radio map. Once the GPS data is determined to be reliable,the GPS location can be added as an extended reference point on theradio map. Thereafter, when a mobile device is in proximity to theextended reference point and obtains RSSI measurements similar to theRSSI measurements obtained by the device at the GPS location, the GPSlocation can be identified as the location of the mobile device.

While a “probabilistic approach” has largely been described as beingused for comparing the location fingerprint stored in the database 106to the RSSI measurements 114 (e.g., the comparing 116 of FIG. 1) todetermine the location of the mobile device 112, other techniques mayalternatively or additionally be used. In some implementations, anearest neighbor test is used in which the RSSI measurements 114 arecompared to the survey data (e.g., the RSSI measurements for each of theAPs 104). The Euclidean distance between the RSSI measurements 114 andeach reference point fingerprint is determined, and the reference pointcorresponding to the smallest Euclidean distance is determined to be thelikely (x, y) location of the mobile device 112.

While the venue has largely been described as being a mall, other venuesmay be surveyed by the survey device to create a radio map to beextended. The venue may be an indoor venue (e.g., a restaurant, ashopping complex, a convention center, an indoor sports or concertstadium, a movie theater, a parking lot, etc.) or an outdoor venue(e.g., a street, an outdoor sports or concert stadium, an amusementpark, a fair, a carnival, a park, a national park, a canyon, a valley, acollection of hiking trails, a parking garage, etc.). In someimplementations, the venue may be aboveground or belowground (e.g., abelowground parking garage or a belowground shopping complex). In someimplementations, the venue is a location that is not able to receivesufficiently accurate GPS signals. Therefore, the venue may be anoutdoor location that includes obstructions to GPS signals (e.g., acrowded city block, a canyon, etc.).

While the RSSI measurements (e.g., for each AP at each reference point,for each AP at each extended reference point, etc.) are largelydescribed as being fit to a Rayleigh distribution, other probabilitydistributions having different probability density functions can also oralternatively be used. For example, in some implementations, one or moreof a Uniform (e.g., Continuous) probability distribution, a Gaussianprobability distribution, and a Ricean probability distribution may beused, among others. In some implementations, one or more aspects of anycombination of the Rayleigh, Uniform, Gaussian, and Ricean probabilitydensity functions may be included in the probability density functionthat is used.

While the radio map (e.g., an initial version of the radio map) has beenlargely described as being obtained by taking RSSI measurements of Wi-Fisignals received from various APs, one or more other wireless protocolsmay be employed instead of or in addition to Wi-Fi. For example, in someimplementations, the survey device may be configured to obtain RSSImeasurements for Bluetooth signals received when the survey device ispositioned at various reference points. The Bluetooth signals may bereceived from various Bluetooth transmitter located throughout and/orproximate to a venue. Such RSSI measurements of the Bluetooth signalsmay be used, either alone or in combination with the Wi-Fi data, togenerate the location fingerprint of the venue.

While the survey data has largely been described as being obtained by asurvey device that measures characteristics of Wi-Fi signals, othertypes of data may be used as “source data.” In other words, survey datais one example of the type of source data that can be used to build theinitial radio map. In general, the source data has a relatively highdegree of accuracy and can be trusted as corresponding to the truelocation of the device. In the examples largely described above, thesurvey data has a relatively high degree of accuracy because thelocations that correspond to each reference point are manually input bya human user. In some implementations, survey data may be “truth data”obtained from truth sources (e.g., sources that are known to providelocation data having a relatively high degree of accuracy, such asuser-input data). In this way, rather than the initial radio map beingbuilt based on survey data, the initial radio map may be built usingother high quality data from other truth sources. In someimplementations, the high quality data may be high quality GPS data(e.g., which may be determined based on the horizontal error associatedwith the GPS data).

While the radio map (e.g., an initial version of the radio map) has beenlargely described as being obtained by a survey device that measures aplurality of RSSIs from various APs at various reference points (e.g.,provided as surveyor-entered positions), the radio map may be obtained(e.g., received) in other ways. In some implementations, the radio mapmay be built from source data other than survey data. In someimplementations, the radio map may be previously obtained andsubsequently extended according to the techniques described herein. Forexample, the radio map may be obtained from a database of radio mapsthat were previously built.

While the radio map has largely been described as being obtained for anindoor venue, a similar process can be applied to build a radio map foran outdoor location. For example, outdoor locations can sometimes relyon GPS data to accurately determine a position of a device. However,some outdoor locations may have characteristics that result ininaccurate position determination using GPS. For example, city streetsmay have surrounding buildings that impede/obscure line of site of GPSsignals, thereby causing difficulty in determining position using GPS.In rural areas, natural barriers (e.g., canyons, valleys, etc.) maysimilarly impede GPS signals. In such locations, a surveying techniquemay be used to building a radio map. Or, for example, one or more othertechniques may be used for building a radio map (e.g., using other truthdata).

Similarly, in some implementations, indoor venues may not require asurveying technique to build a radio map. For example, an indoorlocation may have a glass roof or some other characteristic that allowsGPS signals to sufficiently cover the venue. In such circumstances, GPSdata may be identified as being of relatively high accuracy such thatthe GPS data can be accepted as truth data. In some implementations,such GPS data can be used to build the radio map. In general, outdoorlocations may have characteristics similar to typical indoor locations,and indoor locations may have characteristics similar to typical outdoorlocations, such that the technique described herein as largely applyingto indoor locations can likewise be applied to outdoor locations, andvice versa.

While the radio map has largely been described as being extended byextending a boundary of the radio map, the radio map may be extended inother ways. In general, extending the radio map involves usinginformation related to explored areas of the radio map to determineinformation about unexplored areas of the radio map. For example, theexplored areas of the radio map may represent areas within the venue forwhich a location can be determined at a relatively high level ofaccuracy using survey data. Such locations can be used as anchor points.The anchor points, in combination with additional data (e.g., harvesttrace data), can be used to extend the radio map into unexplored areas.In this way, the radio map can be extended into unexplored areas withoutnecessarily extending a border of the radio map.

In some implementations, an initial radio map may not exist as aprerequisite for extending the radio map. In other words, while we havelargely described an initial radio map being built using survey data anda boundary of the radio map being extended using harvest data, in someimplementations, the initial radio map may be built and subsequentlyextended using harvest data. In some implementations, because such aradio map may not be built based on “truth” data (e.g., source data thatis known to be accurate, such as survey data), such a radio map mayinclude inaccuracies. However, such inaccuracies may be corrected by theiterative process described above with respect to FIG. 4.

In some implementations, the boundary is a soft boundary, such that adegree of blending occurs between reference points and correspondingdata inside and outside the boundary. In some implementations,in-boundary data (e.g., reference points that were generated based ontruth data and that reside within the initial radio map) may be lesssusceptible to modification than data that resides outside of theboundary (e.g., the extended reference points). In some implementations,in-boundary data may not be modified because it is taken as truth data.In other words, the system may keep intact in-boundary data because itwas obtained under circumstances that ensure data of high accuracy.

In some implementations, the WLAN (e.g., Wi-Fi) infrastructure mayfollow an IEEE standard, such as an IEEE 802.11 protocol, although otherprotocols may also or alternatively be used.

This disclosure describes various Graphical User Interfaces (UIs) forimplementing various features, processes or workflows. These GUIs can bepresented on a variety of electronic devices including but not limitedto laptop computers, desktop computers, computer terminals, televisionsystems, tablet computers, e-book readers and smart phones. One or moreof these electronic devices can include a touch-sensitive surface. Thetouch-sensitive surface can process multiple simultaneous points ofinput, including processing data related to the pressure, degree orposition of each point of input. Such processing can facilitate gestureswith multiple fingers, including pinching and swiping.

When the disclosure refers “to select” or “selecting” user interfaceelements in a GUI, these terms are understood to include clicking or“hovering” with a mouse or other input device over a user interfaceelement, or touching, tapping or gesturing with one or more fingers orstylus on a user interface element. User interface elements can bevirtual buttons, menus, selectors, switches, sliders, scrubbers, knobs,thumbnails, links, icons, radial buttons, checkboxes and any othermechanism for receiving input from, or providing feedback to a user.

Example 2D Feature-Based SLAM

FIG. 7 is a block diagram of another exemplary system 700 for extendinga radio map using feature-based SLAM. System 700 includes harvest tracecollector 701, measurement constraint processor 702, loop constraintprocessor 703, landmark constraint processor 704, non-linear leastsquares (NLS) solver 705 and map builder 706. In an embodiment, system700 detects features in the environment to extend radio map coverageinto areas where survey data is unavailable.

I. 2D Feature-Based SLAM—Problem Statement

In an embodiment, system 700 determines the best estimate of statesZ=[z₀, z₁, z₂ . . . z_(n)] that maximizes the posterior probability of Zgiven measurements U=[u₀, u₁ . . . u_(n-1)], landmarks L=[{l₁, z_(i1)},{l_(2,z) _(i2) } . . . {l_(m), z_(im)}] and loop constraints C=[c₀, c₁,. . . c_(t)] (hereinafter referred to collectively as locationconstraints):

p(Z|U,L,C)∝p(Z|U)p(L|Z)p(C|Z).

Z can be obtained by harvest trace collector 701 from a single trace ormultiple traces. If all the states z_(i)∈Z are from a single trace, thenmeasurement constraint processor 702 calculates the posteriorprobability P(Z|U) as shown below using the log function to simplify theexpression.

$\mspace{20mu} \left. \begin{matrix}{{P\left( Z \middle| U \right)} \propto {p\left( {\text{?}\left| \text{?} \middle| {\text{?}\mspace{14mu} \ldots \mspace{14mu} \text{?}} \right|\text{?}} \right.}} \\{\propto {e\text{?}}}\end{matrix}\mspace{20mu}\Rightarrow{{\log \; {P\left( Z \middle| U \right)}} \propto {\text{?}\left( {\text{?}\left( \text{?} \right)} \right)\text{?}\text{?}\left( {z_{k} - {f\left( \text{?} \right)}} \right)}} \right.$?indicates text missing or illegible when filed

If the states z_(i)∈Z are from different traces, then measurementconstraint processor 702 calculates the posterior probability P(Z|U) asshown below.

$\mspace{20mu} \begin{matrix}{{P\left( Z \middle| U \right)} \propto {\text{?}{P\left( Z_{i} \middle| U_{i} \right)}}} \\{\propto {\text{?}\text{?}{p\left( \text{?} \middle| \text{?} \right)}}}\end{matrix}$ $\mspace{20mu} \begin{matrix}\left. \Rightarrow{{{logP}\left( Z \middle| U \right)} \propto {\text{?}\log \; {P\left( Z_{i} \middle| U_{i} \right)}}} \right. \\{\propto {\text{?}\text{?}\left( {\text{?} - {j\left( \text{?} \right)}} \right)\text{?}\text{?}\left( {\text{?} - {f\left( \text{?} \right)}} \right)}}\end{matrix}$ ?indicates text missing or illegible when filed

Loop constraint processor 703 calculates the posterior probabilityP(C|Z) as shown below using the log function to simplify the expression.

$\left. {{P\left( C \middle| Z \right)} \propto {\prod\limits_{C_{ij} \in C}\; {p\left( {\left. C_{ij} \middle| z_{i} \right.,z_{j}} \right)}} \propto e^{\sum\limits_{C_{ij}}{{- \frac{1}{2}}{({z_{j} - z_{i}})}^{T}{R_{C_{ij}}^{- 1}{({z_{j} - z_{i}})}}}}}\Rightarrow{{\log \left( {P\left( C \middle| Z \right)} \right)} \propto {\sum\limits_{C_{ij}}{\left( {z_{j} - z_{i}} \right)^{T}{R_{C_{ij}}^{- 1}\left( {z_{j} - z_{i}} \right)}}}} \right.$

Landmark constraint processor 704 calculates the posterior probabilityP(L|Z) as shown below using the log function to simplify the expression.

$\left. {{P\left( L \middle| Z \right)} \propto {\prod\limits_{{\{{l,z}\}} \in L}\; {p\left( \left\{ {l,z} \right\} \middle| z \right)}} \propto e^{\sum\limits_{\{{l,z}\}}{{- \frac{1}{2}}{({z_{j} - L_{k}})}^{T}{R_{L_{k}}^{- 1}{({z_{j} - L_{k}})}}}}}\Rightarrow{{\log \left( {P\left( L \middle| Z \right)} \right)} \propto {\sum\limits_{{\{{l,z}\}} \in L}{\left( {z_{j} - L_{k}} \right)^{T}{R_{L_{k}}^{- 1}\left( {z_{j} - L_{k}} \right)}}}} \right.$

The posterior probabilities described above can be summed together toget log p(Z|U, L, C). Log p(Z|U, L, C) is input into a NLS solver 705(e.g., Gaussian-Newton, Levenberg-Marquardt) to find the best estimateof Z. The output of NLS solver 705 is a global best estimate of tracelocations or the best estimate of Z. The WiFi signals observed are thenreferenced to those updated locations and used in radio map builder 706.

II. Landmark and Loop Constraints Detection

Typically in a two-dimensional (2D) x-y plane, landmark and loopconstraints only indicate the proximity of x (latitude) and y(longitude), not θ (heading). Accordingly, the heading θ is ignored whencomputing the residual between states Z and landmark and loopconstraints. In an embodiment, harvest trace collector 701 collectsharvest traces and their corresponding timestamps. The harvest tracesinclude trace data, such as pedometer data with speed and attitudeinformation, pressure data which can indicate relative altitude changewithin a trace, WiFi scan data which is captured by a wirelesstransceiver along the trace and a location fix that includes latitudeand longitude estimated from a location processor. The trace data can beused by the system to extract landmark and loop constraints to helpsolve a 2D graph simultaneous localization and mapping (SLAM) problem.

A. Location Landmark Constraint

In an embodiment, a location fix (latitude, longitude, uncertainty)provided by a location processor (e.g., a GPS receiver, WiFi positionestimator) is used as a landmark, and the uncertainty is used as acovariance matrix of the landmark.

B. WiFi Hotspot Loop Constraint

The WiFi scan data contains a timestamp, AP MAC address and an observedRSSI value. In an embodiment, the system can determine by the RSSIvalues when a trace encounters a very strong WiFi AP signal, such as aWiFi hotspot. If two states from two different traces (or from the sametrace if not too close in time) observe the same strong WiFi AP signal,the system assumes the WiFi APs are physically close to each other, anda WiFi hotspot loop constraint is created. In an embodiment, if aparticular WiFi AP with observed RSSI values greater than a particularRSSI threshold (e.g., −40 dBm), the particular WiFi AP is a strong WiFiAP, and a fixed covariance is used for the WiFi loop constraint. A WiFihotspot is generally less noisy than other loop constraints, but itsuffers from an availability issue. For example, a trace typically doesnot have a WiFi hotspot observation very often and many traces neverobserve a WiFi hotspot.

C. WiFi Scan Similarity Loop Constraint

In an embodiment, instead or in addition to using a WiFi hotspot loopconstraint, a WiFi scan similarity loop constraint can be used tocompare APs from two different WiFi scans to find similar WiFi scanpairs. The steps to detect WiFi scan pairs includes grouping the WiFiscan data into an N-second (e.g., N=5 s) time window, and consideringeach group as a WiFi scan event. A WiFi scan distance between a WiFiscan pair can be computed by combining the WiFi scan Jaccard distanceand Euclidean distance. Any WiFi scan pair whose distance is lower thana threshold distance T (e.g., T<4.5) generates a WiFi scan similarityloop constraint. A fixed covariance is used for the WiFi scan similarityconstraints. WiFi scan similarity loop constraints are more noisy thenother loop constraints, but there are typically many WiFi scans pertrace. A larger fixed covariance can be used to compensate for theincreased noise.

D. Floor Transition Loop Constraint

In a multi-level indoor environment, a single harvest trace may havefloor transition events, which can be detected using barometer pressuredata. In an embodiment, the system can determine when harvest tracesoverlap with each other during floor transitions by comparing the WiFiscan data that was captured during the floor transition and use the datato create a floor transition loop constraint. In an embodiment, thesteps to create a floor transition loop constraint include: 1) detectinga floor transition period for each harvest trace using an offline floortransition detector, 2) computing a normalized WiFi scan distance foreach WiFi AP pair using dynamic time warping and the WiFi scan distancefunction described above (e.g., by combining the WiFi scan Jaccarddistance and Euclidean distance); and 3) determining for any floortransition the WiFi AP pair whose WiFi scan distance is lower than adistance threshold, and generating the floor transition loop constraint.

Similar to the other loop constraints, a fixed covariance is used forthe floor transition constraint. The floor transition loop constraint isless noisy then the WiFi scan similarity loop constraint, but noisierthen the WiFi hotspot loop constraint. The floor transition loopconstraint also suffers from an availability issue as not every tracehas a floor transition event. Floor transition events can be used forgraph SLAM on the z-axis (i.e., altitude, floor ordinal). In anembodiment, the z-axis can be added to the 2D graph SLAM, athree-dimensional (3D) graph SLAM can be created, or a 1D graph SLAM canbe used on the z-axis only.

III. State Transition Model Description and Inputs

Having described the problem setup and landmark/loop constraintdetections, a more detailed computation of the posterior probabilitieswill now be described.

1. State: A user location trace can be described by a set of k statesz_(k)=[x_(k), y_(k), θ_(k)]^(T), where x_(k) is latitude, y_(k) islongitude and θ_(k) is heading.2. Measurement: Between each consecutive states from a single trace,there is also a measurement u_(k)=[Δd_(k), Δθ_(k)]^(T), where: Δd_(k) isthe distance between two consecutive states, Δθ_(k) is the headingchange between two consecutive states and the state transition model isgiven by z_(k)=f (z_(k-1), u_(k-1))+n_(k), where n_(k) is the statetransition model noise, and we assume the noise follows a zero meanGaussian distribution with covariance P_(k): n_(k)˜N(0, P_(k)). Theequations of motion are as follows.

x _(k) =x _(k-1) +Δd _(k-1) cos(θ_(k-1)+Δθ_(k-1))+n _(k) ^(x)

y _(k) =y _(k-1) +Δd _(k-1) sin(θ_(k-1)+Δθ_(k-1))+n _(k) ^(y)

θ_(k)=θ_(k-1)+Δθ_(k-1) +n _(k) ^(θ)

Besides the state transition model, the measurement also has noise. Weassume the noise of u_(k) follows a zero mean Gaussian distribution withcovariance Q_(k). Due to noise in the measurement and state transitionmodel, there will be error between the predicted state f(z_(k)−1,u_(k)−1) and the true state z_(k). We assume this error also follows azero mean multivariate Gaussian distribution with covariance w_(k-1).The covariance w_(k-1) is computed as follows.

${w_{k - 1} = {{A_{k - 1}Q_{k - 1}A_{k - 1}^{T}} + P_{k - 1}}},{{where}\mspace{14mu} A_{k - 1}\mspace{14mu} {is}\mspace{14mu} {given}\mspace{14mu} {by}{\text{:}\begin{bmatrix}{\cos \left( {\theta_{k - 1} + {\Delta \; \theta_{k - 1}}} \right)} & {{- \Delta}\; d_{k - 1}{\sin \left( {\theta_{k - 1} + {\Delta \; \theta_{k - 1}}} \right)}} \\{\sin \left( {\theta_{k - 1} + {\Delta \; \theta_{k - 1}}} \right)} & {\Delta \; d_{k - 1}{\cos \left( {\theta_{k - 1} + {\Delta \; \theta_{k - 1}}} \right)}} \\0 & 1\end{bmatrix}}}$

The posterior probability of the state z_(k) with measurement u_(k-1)and a previous state z_(k-1) can be computed as follows.

${p\left( {\left. z_{k} \middle| z_{k - 1} \right.,u_{k - 1}} \right)} \propto e^{{- \frac{1}{2}}{({z_{k} - {f{({z_{k - 1},u_{k - 1}})}}})}^{T}{w_{k - 1}^{- 1}{({z_{k} - {f{({z_{k - 1},u_{k - 1}})}}})}}}$

3. Landmarks: Landmarks can be observed in the environment with knownlocation L_(k)=[x_(L) _(k) , y_(L) _(k) ]^(T), such as, for example, aGPS location from harvest data. Let {L_(k), z_(j)} denote observinglandmark L_(k) at/near state z_(j), then the residual isz_(j)−L_(k)=n_(k). The landmark observation can have some noise n_(k),which is assumed to follow a zero mean multivariate Gaussiandistribution: n_(k)˜N(0, R_(L) _(k) ), where R_(L) _(k) is thecovariance matrix for measurement L_(k). Then the posterior probabilityof the landmark {L_(k), z_(j)} given state z_(j) is as follows.

${p\left( \left\{ {L_{k},z_{j}} \right\} \middle| z_{j} \right)} \propto e^{{- \frac{1}{2}}{({z_{j} - L_{k}})}^{T}{R_{L_{k}}^{- 1}{({z_{j} - L_{k}})}}}$

4. Loop Constraints: Loop constraints C_(ij) can also be observed in theenvironment that indicate two states z_(i) and z_(j) that are physicallyclose to each other, i.e., z_(j)−z_(i)=n_(ij). The loop constraints canhave noise n_(ij), and it is assumed the noise follows a zero meanmultivariate Gaussian distribution: n_(ij)˜N(0, R_(C) _(ij) ), whereR_(C) _(ij) is the covariance matrix for loop constraint C_(ij). Thenthe posterior probability of the loop constraint C_(ij) given statesz_(i) and z_(j) is as follows.

${p\left( {\left. C_{ij} \middle| z_{i} \right.,z_{j}} \right)} \propto e^{{- \frac{1}{2}}{({z_{j} - z_{i}})}^{T}{R_{C_{ij}}^{- 1}{({z_{j} - z_{i}})}}}$

The landmark and loop constraints described above may not apply to allelements of the states (e.g., landmark and physically close loopconstraints only apply to x_(k) and y_(k)). In these cases, an openconstraint on one of the states (e.g., θ) could be expressed by a largecovariance in the corresponding element in R_(L) _(k) , R_(C) _(ij) , orexpressed with a transform matrix like H=[1 0 0; 0 1 0]. Then,H*z_(j)−L_(k)=n_(k) or H*(z_(i)−z_(j))=n_(ij).

FIG. 8A is a flowchart of another exemplary process 800 of extending aradio map using feature-based SLAM. Process 800 can be implemented usingthe system architecture described in reference to FIG. 9.

Process 800 can begin by receiving, by one or more processors, firsttrace data from a first mobile device operating in an environment (801).The first set of trace data describes a first set of states of a firsttrajectory of the first mobile device (e.g., a smartphone) in theenvironment. The first set of trace data includes a first set ofwireless access point data (e.g., WiFi scan data) and a first set ofsensor measurements (e.g., acceleration data, angular rate data,pressure data) collected by the first mobile device while operating inthe environment (e.g., an indoor environment, dense urban environment).

Process 800 continues by receiving, by the one or more processors, asecond set of trace data from a second mobile device operating in theenvironment (802). The second set of trace data describes a second setof states of a second trajectory of the second mobile device in theenvironment. The second set of trace data includes a second set ofwireless access point data and a second set of sensor measurementscollected by the second mobile device while operating in theenvironment. In an embodiment, the first and second mobile device arethe same mobile device, and the in other embodiments, the first andsecond mobile devices are different mobile devices.

Process 800 continues by determining, by the one or more processors, afirst location constraint (e.g., a measurement constraint) based on atleast one of the first or second set of sensor measurements (803) (e.g.,delta distance or delta heading measurements).

Process 800 continues by determining, by the one or more processors, asecond location constraint based on at least one of signal strengthmeasurements (e.g., RSSI values) in the first or second sets of wirelessaccess point data (e.g., WiFi scans) or a similarity between the firstand second sets of wireless access point data (804).

Process 800 continues by generating, by the one or more processors, aradio map based on the first and second constraints (805), as describedin reference to FIGS. 1-7.

Example Feature-Based SLAM with Z-Axis Location

In some applications, there is a need to survey indoor structures (e.g.,a multi-story building) that have a number of floors when there is noprior knowledge of the indoor structure. In an embodiment, a floortransition algorithm is applied to trace data of multiple tracesharvested from mobile devices operating in the indoor structure todetect one or more floor transitions in the traces. Each trace is brokendown into multiple single floor traces (also referred to as “floornodes”) based on the detected floor transitions, where each floor traceis considered a floor node. Barometric pressure data is used todetermine the delta distance between floor nodes. The delta distance(e.g., in meters) can be converted into a floor ordinal (e.g., aninteger value) by comparing the delta distance to a distance threshold(e.g., 3 meters) to determine the number of floor transitions betweenfloor nodes. In an embodiment, the floor ordinal can then be used as anadditional feature with the 2D feature-based SLAM described in referenceto FIGS. 7 and 8, to compare different trajectories using anoptimization solver and a number of location constraints (e.g.,landmark, WiFi scan similarity, WiFi hot spot).

I. Z-Axis 1D Feature Based SLAM—Problem Statement

In an embodiment, system 700 determines the best estimate of statesZ=[z₀, z₁, z₂ . . . z_(n)] that maximizes the posterior probability of Zgiven measurements U=[u₀, u₁ . . . u_(n-1)], landmarks L=[{l₁, z_(i1)},{l_(2,z) _(i2) } . . . {l_(m), z_(im)}] and loop constraints C=[c₀, c₁,. . . c_(t)]:

p(Z|U,L,C)∝p(Z|U)p(L|Z)p(C|Z).

Like in the previously described 2D SLAM embodiment, Z can be obtainedby harvest trace collector 701 from a single trace or multiple traces.If all the states z_(i)∈Z are from a single trace, then measurementconstraint processor 702 calculates the posterior probability P(Z|U) asshown below using the log function to simplify the expression.

$\left. \begin{matrix}{{P\left( Z \middle| U \right)} \propto {{p\left( {\left. z_{1} \middle| z_{0} \right.,u_{0}} \right)}{p\left( {\left. z_{2} \middle| z_{1} \right.,u_{1}} \right)}\mspace{14mu} \ldots \mspace{14mu} {p\left( {\left. z_{n} \middle| z_{n - 1} \right.,u_{n - 1}} \right)}}} \\{\propto e^{\sum_{z_{k} \in Z}{- \frac{{({z_{k} - {f{({z_{k - 1},u_{k - 1}})}}})}^{2}}{\delta_{k - 1}^{2}}}}}\end{matrix}\Rightarrow{{\log \; {P\left( Z \middle| U \right)}} \propto {\sum\limits_{z_{k} \in Z}\frac{\left( {z_{k} - {f\left( {z_{k - 1},u_{k - 1}} \right)}} \right)^{2}}{\delta_{k - 1}^{2}}}} \right.,$

where δ_(k-1) ² is a variance.

If the states z_(i)∈Z are from different traces, then measurementconstraint processor 702 calculates the posterior probability P(Z|U) asshown below.

$\left. \begin{matrix}{{{P\left( Z \middle| U \right)} \propto {\prod\limits_{Z_{i} \in Z}\; {P\left( Z_{i} \middle| U_{i} \right)}}},} \\{{\propto {\prod\limits_{Z_{i} \in Z}{\prod\limits_{z_{k} \in Z_{i}}{p\left( {\left. z_{k} \middle| z_{k - 1} \right.,u_{k - 1}} \right)}}}},}\end{matrix}\Rightarrow{{\log \; {P\left( Z \middle| U \right)}} \propto {\sum\limits_{Z_{i} \in Z}{\sum\limits_{z_{k} \in Z_{i}}\frac{\left( {z_{k} - {f\left( {z_{k - 1},u_{k - 1}} \right)}} \right)^{2}}{\delta_{k - 1}^{2}}}}} \right.$

Loop constraint processor 703 calculates the posterior probabilityP(C|Z) as shown below using the log function to simplify the expression.

$\left. {{P\left( C \middle| Z \right)} \propto {\prod\limits_{C_{ij} \in C}\; {p\left( {\left. C_{ij} \middle| z_{i} \right.,z_{j}} \right)}} \propto e^{\sum\limits_{C_{ij}}\frac{{- \frac{1}{2}}{({z_{j} - z_{i}})}^{2}}{\delta_{c_{ij}}^{2}}}}\Rightarrow{{\log \left( {P\left( C \middle| Z \right)} \right)} \propto {\sum\limits_{C_{ij}}\frac{\left( {z_{j} - z_{i}} \right)^{2}}{\delta_{c_{ij}}^{2}}}} \right.$

Landmark constraint processor 704 calculates the posterior probabilityP(L|Z) as shown below using the log function to simplify the expression.

$\left. {{P\left( L \middle| Z \right)} \propto {\prod\limits_{{\{{l,z}\}} \in L}\; {p\left( \left\{ {l,z} \right\} \middle| z \right)}} \propto e^{\sum\limits_{\{{l,z}\}}{- \frac{{({z_{j} - L_{k}})}^{2}}{2\delta_{L_{k}}^{2}}}}}\Rightarrow{{\log \left( {P\left( L \middle| Z \right)} \right)} \propto {\sum\limits_{\{{l,z}\}}{\frac{\left( {z_{j} - L_{k}} \right)^{2}}{\delta_{L_{k}}^{2}}.}}} \right.$

II. State Transition Model Description and Inputs

Having described the problem setup and landmark/loop constraintdetections, a more detailed computation of the posterior probabilitieswill now be described.

1. State: A user location trace can be described by a set of k statesz_(k), where z_(k) is the floor ordinal.2. Measurement: Between each consecutive states from a single trace,there is also a measurement Δz_(k), which is the change in floor ordinalbetween two consecutive states and the state transition model is givenby z_(k)=f(z_(k-1), u_(k-1))+n_(k), where n_(k) is the state transitionmodel noise, and we assume the noise follows a zero mean Gaussiandistribution with variance δ_(k) ²: n_(k)˜N(0, δ²). The equation ofmotion is as follows.

z _(k) =z _(k-1) +Δz _(k-1) +n _(k)

The posterior probability of the state z_(k) with measurement u_(k-1)and a previous state z_(k-1) can be computed as follows.

${p\left( {\left. z_{k} \middle| z_{k - 1} \right.,u_{k - 1}} \right)} \propto e^{{- \frac{1}{2}}\frac{{({z_{k} - {f{({z_{k - 1},u_{k - 1}})}}})}^{2}}{\delta_{k - 1}^{2}}}$

3. Landmarks: Landmarks can be observed in the environment with knownz-axis location or floor ordinal L_(k)=z_(L) _(k) , such as, forexample, a GPS location from harvest data. Let {L_(k), z_(j)} denoteobserving landmark L_(k) at/near state z_(j), then the residual isz_(j)−L_(k)=n_(k). The landmark observation can have some noise n_(k),which is assumed to follow a zero mean Gaussian distribution: n_(k)˜N(0,δ_(L) _(k) ²), where δ_(L) _(k) ² is the variance for measurement L_(k).Then the posterior probability of the landmark {L_(k), z_(j)} givenstate z_(j) is as follows.

${p\left( \left\{ {L_{k},z_{j}} \right\} \middle| z_{j} \right)} \propto e^{- \frac{{({z_{j} - L_{k}})}^{2}}{2\delta_{L_{k}}^{2}}}$

4. Loop Constraints: Loop constraints C_(ij) can also be observed in theenvironment that indicate two states z_(i) and z_(j) that are physicallyclose to each other on the same floor ordinal, i.e., z_(j)−z_(i)=n_(ij).The loop constraints can have noise n_(ij), and it is assumed the noisefollows a zero mean Gaussian distribution: n_(ij)˜N(0, δ_(C) _(ij) ²),where δ_(C) _(ij) ² is the variance for loop constraint C_(ij). Then theposterior probability of the loop constraint C_(ij) given states z_(i)and z_(j) is as follows.

${p\left( {\left. C_{ij} \middle| z_{i} \right.,z_{j}} \right)} \propto e^{- \frac{{({z_{j} - z_{i}})}^{2}}{2\delta_{C_{ij}}^{2}}}$

FIG. 8B is a flowchart of another exemplary process 806 of extending aradio map using feature-based SLAM with z-axis location. Process 806 canbe implemented by the system architecture 900 described in reference toFIG. 9.

Process 806 begins by determining floor transitions in mobile devicetrajectories based on pressure measurements (807). For example, pressuresensors in mobile devices can collect time-stamped pressure measurementsusing, for example, a barometer. The pressure measurements can beincluded in trace data sent by the mobile devices to a server forfurther processing. In an embodiment, floor transitions are determinedusing a change point detection algorithm for time series data, such as,for example, the algorithm described in Killick, R., Fearnhead, P. andEckley, I. A., “Optimal Detection of Changepoints With a LinearComputational Cost,” Journal of the American Statistical Association107(500):1590-1598 (December 2012). The time series of pressure data isassumed to follow a Gaussian distribution with a very small variance anda changing mean. The mean is constant if the user is on the same floorand will only change when the user transitions to another floor. Withthis assumption, the change point detection algorithm computes a costfunction for each change point candidate and picks the appropriatechange point candidate.

Process 806 continues by dividing the trajectories into floor nodesbased on the floor transitions (808), determining the delta distancebetween the floor nodes (809), converting the delta distance into floorordinals (810) and then using the floor ordinals as additional featuresin 2D feature-based SLAM to compare the device trajectories usingoptimization solver and location (811). For example, in an embodimentbarometric pressure data is used to determine the delta distance betweenfloor nodes. The delta distance is converted into a floor ordinal bycomparing the delta distance to a distance threshold to determine thenumber of floor transitions between floor nodes. The floor ordinal isthen used as an additional feature with the 2D feature-based SLAM (seeFIGS. 7 and 8) to compare different trajectories using an optimizationsolver and a number of location constraints (e.g., measurementconstraints, landmark constraints, loop constraints).

Example System Architecture

FIG. 9 is a block diagram of an exemplary system architecture of anelectronic device implementing the features and processes of FIGS. 1-8.The architecture 900 can be implemented on any electronic device thatruns software applications derived from compiled instructions, includingwithout limitation personal computers, servers, smart phones, mediaplayers, electronic tablets, game consoles, email devices, etc. In someimplementations, the architecture 900 can include one or more processors902, one or more input devices 904, one or more display devices 906, oneor more network interfaces 908 and one or more computer-readable mediums910. Each of these components can be coupled by bus 912.

Display device 906 can be any known display technology, including butnot limited to display devices using Liquid Crystal Display (LCD) orLight Emitting Diode (LED) technology. Processor(s) 902 can use anyknown processor technology, including but are not limited to graphicsprocessors and multi-core processors.

Input device 904 can be any known input device technology, including butnot limited to a keyboard (including a virtual keyboard), mouse, trackball, and touch-sensitive pad or display. In some implementations, theinput device 904 could include a microphone that facilitatesvoice-enabled functions, such as speech-to-text, speaker recognition,voice replication, digital recording, and telephony functions. The inputdevice 904 can be configured to facilitate processing voice commands,voiceprinting and voice authentication. In some implementations, audiorecorded by the input device 904 is transmitted to an external resourcefor processing. For example, voice commands recorded by the input device904 may be transmitted to a network resource such as a network serverwhich performs voice recognition on the voice commands.

Bus 912 can be any known internal or external bus technology, includingbut not limited to ISA, EISA, PCI, PCI Express, NuBus, USB, Serial ATAor FireWire.

Computer-readable medium 910 can be any medium that participates inproviding instructions to processor(s) 902 for execution, includingwithout limitation, non-volatile storage media (e.g., optical disks,magnetic disks, flash drives, etc.) or volatile media (e.g., SDRAM, ROM,etc.).

Computer-readable medium 910 can include various instructions 914 forimplementing an operating system (e.g., Mac OS®, Windows®, Linux). Theoperating system can be multi-user, multiprocessing, multitasking,multithreading, real-time and the like. The operating system performsbasic tasks, including but not limited to: recognizing input from inputdevice 904; sending output to display device 906; keeping track of filesand directories on computer-readable medium 910; controlling peripheraldevices (e.g., disk drives, printers, etc.) which can be controlleddirectly or through an I/O controller; and managing traffic on bus 912.Network communications instructions 916 can establish and maintainnetwork connections (e.g., software for implementing communicationprotocols, such as TCP/IP, HTTP, Ethernet, etc.).

A graphics processing system 918 can include instructions that providegraphics and image processing capabilities. For example, the graphicsprocessing system 918 can implement the processes described withreference to FIGS. 1-8.

Application(s) 920 can be an application that uses or implements theprocesses described in reference to FIGS. 1-8. The processes can also beimplemented in operating system 914.

The described features can be implemented advantageously in one or morecomputer programs that are executable on a programmable system includingat least one programmable processor coupled to receive data andinstructions from, and to transmit data and instructions to, a datastorage system, at least one input device, and at least one outputdevice. A computer program is a set of instructions that can be used,directly or indirectly, in a computer to perform a certain activity orbring about a certain result. A computer program can be written in anyform of programming language (e.g., Objective-C, Java), includingcompiled or interpreted languages, and it can be deployed in any form,including as a stand-alone program or as a module, component,subroutine, or other unit suitable for use in a computing environment.

Suitable processors for the execution of a program of instructionsinclude, by way of example, both general and special purposemicroprocessors, and the sole processor or one of multiple processors orcores, of any kind of computer. Generally, a processor will receiveinstructions and data from a read-only memory or a random access memoryor both. The essential elements of a computer are a processor forexecuting instructions and one or more memories for storing instructionsand data. Generally, a computer will also include, or be operativelycoupled to communicate with, one or more mass storage devices forstoring data files; such devices include magnetic disks, such asinternal hard disks and removable disks; magneto-optical disks; andoptical disks. Storage devices suitable for tangibly embodying computerprogram instructions and data include all forms of non-volatile memory,including by way of example semiconductor memory devices, such as EPROM,EEPROM, and flash memory devices; magnetic disks such as internal harddisks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROMdisks. The processor and the memory can be supplemented by, orincorporated in, ASICs (application-specific integrated circuits).

To provide for interaction with a user, the features can be implementedon a computer having a display device such as a CRT (cathode ray tube)or LCD (liquid crystal display) monitor for displaying information tothe user and a keyboard and a pointing device such as a mouse or atrackball by which the user can provide input to the computer.

The features can be implemented in a computer system that includes aback-end component, such as a data server, or that includes a middlewarecomponent, such as an application server or an Internet server, or thatincludes a front-end component, such as a client computer having agraphical user interface or an Internet browser, or any combination ofthem. The components of the system can be connected by any form ormedium of digital data communication such as a communication network.Examples of communication networks include, e.g., a LAN, a WAN, and thecomputers and networks forming the Internet.

The computer system can include clients and servers. A client and serverare generally remote from each other and typically interact through anetwork. The relationship of client and server arises by virtue ofcomputer programs running on the respective computers and having aclient-server relationship to each other.

One or more features or steps of the disclosed embodiments can beimplemented using an API. An API can define on or more parameters thatare passed between a calling application and other software code (e.g.,an operating system, library routine, function) that provides a service,that provides data, or that performs an operation or a computation.

The API can be implemented as one or more calls in program code thatsend or receive one or more parameters through a parameter list or otherstructure based on a call convention defined in an API specificationdocument. A parameter can be a constant, a key, a data structure, anobject, an object class, a variable, a data type, a pointer, an array, alist, or another call. API calls and parameters can be implemented inany programming language. The programming language can define thevocabulary and calling convention that a programmer will employ toaccess functions supporting the API.

In some implementations, an API call can report to an application thecapabilities of a device running the application, such as inputcapability, output capability, processing capability, power capability,communications capability, etc.

A number of implementations have been described. Nevertheless, it willbe understood that various modifications may be made. For example, othersteps may be provided, or steps may be eliminated, from the describedflows, and other components may be added to, or removed from, thedescribed systems. Accordingly, other implementations are within thescope of the following claims.

Example Mobile Device Architecture

FIG. 10 is a block diagram of an exemplary device architecture of acomputing device 1000, such as a mobile device, that can implement thefeatures and operations described in reference to FIGS. 1-8. Forexample, the survey device (102 of FIG. 1) and/or the mobile device (112of FIG. 1) may be examples of the computing device 1000. The computingdevice 1000 can include a memory interface 1002, one or more dataprocessors, image processors and/or central processing units 1004, and aperipherals interface 1006. The memory interface 1002, the one or moreprocessors 1004 and/or the peripherals interface 1006 can be separatecomponents or can be integrated in one or more integrated circuits. Thevarious components in the computing device 1000 can be coupled by one ormore communication buses or signal lines.

Sensors, devices, and subsystems can be coupled to the peripheralsinterface 1006 to facilitate multiple functionalities. For example,motion sensor(s) 1010 (e.g., accelerometer, gyroscope, magnetometer), alight sensor 1012, and a proximity sensor 1014 can be coupled to theperipherals interface 1006 to facilitate orientation, lighting, andproximity functions. Other sensors 1016 can also be connected to theperipherals interface 1006, such as a global navigation satellite system(GNSS) (e.g., GPS receiver), a temperature sensor, barometric pressuresensor, a biometric sensor or other sensing device, to facilitaterelated functionalities. The motion sensor(s) 1010 can provide sensordata (e.g., acceleration data, angular rate data) to a digital pedometerapplication for counting steps or step frequency. A magnetometer canprovide input (e.g., magnetic north) to a compass application that cancompute a heading or direction of device travel.

A camera subsystem 1020 and an optical sensor 1022, e.g., a chargedcoupled device (CCD) or a complementary metal-oxide semiconductor (CMOS)optical sensor, can be utilized to facilitate camera functions, such asrecording photographs and video clips. The camera subsystem 1020 and theoptical sensor 1022 can be used to collect images of a user to be usedduring authentication of a user, e.g., by performing facial recognitionanalysis.

Communication functions can be facilitated through one or more wirelesscommunication subsystems 1024, which can include radio frequencyreceivers and transmitters and/or optical (e.g., infrared) receivers andtransmitters. The specific design and implementation of thecommunication subsystem 1024 can depend on the communication network(s)over which the computing device 1000 is intended to operate. Forexample, the computing device 1000 can include communication subsystems1024 designed to operate over a GSM network, a GPRS network, an EDGEnetwork, a WiFi or WiMax network, and a Bluetooth™ network. Inparticular, the wireless communication subsystems 1024 can includehosting protocols such that the device 1000 can be configured as a basestation for other wireless devices. The wireless communicationsubsystems 1024 can receive radio frequency (RF) signals from wirelessnetwork access points (e.g., WiFi scans), cell towers or other devicesand compute signal strength metrics, such as, for example, RSSI values.

An audio subsystem 1026 can be coupled to a speaker 1028 and amicrophone 1030 to facilitate voice-enabled functions, such as speakerrecognition, voice replication, digital recording, and telephonyfunctions. The audio subsystem 1026 can be configured to facilitateprocessing voice commands, voiceprinting and voice authentication. Insome implementations, the microphone 1030 facilitates voice-enabledfunctions, such as speech-to-text, speaker recognition, voicereplication, digital recording, and telephony functions. The audiosubsystem 1026 can be configured to facilitate processing voicecommands, voiceprinting and voice authentication. In someimplementations, audio recorded by the audio subsystem 1026 istransmitted to an external resource for processing. For example, voicecommands recorded by the audio subsystem 1026 may be transmitted to anetwork resource such as a network server which performs voicerecognition on the voice commands.

The I/O subsystem 1040 can include a touch-surface controller 1042and/or other input controller(s) 1044. The touch-surface controller 1042can be coupled to a touch surface 1046. The touch surface 1046 andtouch-surface controller 1042 can, for example, detect contact andmovement or break thereof using any of a plurality of touch sensitivitytechnologies, including but not limited to capacitive, resistive,infrared, and surface acoustic wave technologies, as well as otherproximity sensor arrays or other elements for determining one or morepoints of contact with the touch surface 1046.

The other input controller(s) 1044 can be coupled to other input/controldevices 1048, such as one or more buttons, rocker switches, thumb-wheel,infrared port, USB port, and/or a pointer device such as a stylus. Theone or more buttons (not shown) can include an up/down button for volumecontrol of the speaker 1028 and/or the microphone 1030.

In one implementation, a pressing of the button for a first duration candisengage a lock of the touch surface 1046; and a pressing of the buttonfor a second duration that is longer than the first duration can turnpower to the computing device 1000 on or off. Pressing the button for athird duration can activate a voice control, or voice command, modulethat enables the user to speak commands into the microphone 1030 tocause the device to execute the spoken command. The user can customize afunctionality of one or more of the buttons. The touch surface 1046 can,for example, also be used to implement virtual or soft buttons and/or akeyboard.

In some implementations, the computing device 1000 can present recordedaudio and/or video files, such as MP3, AAC, and MPEG files. In someimplementations, the computing device 1000 can include the functionalityof an MP3 player, such as an iPod™. The computing device 1000 can,therefore, include a 36-pin connector that is compatible with the iPod.Other input/output and control devices can also be used.

The memory interface 1002 can be coupled to memory 1050. The memory 1050can include high-speed random access memory and/or non-volatile memory,such as one or more magnetic disk storage devices, one or more opticalstorage devices, and/or flash memory (e.g., NAND, NOR). The memory 1050can store an operating system 1052, such as Darwin, RTXC, LINUX, UNIX,OS X, WINDOWS, or an embedded operating system such as VxWorks.

The operating system 1052 can include instructions for handling basicsystem services and for performing hardware dependent tasks. In someimplementations, the operating system 1052 can be a kernel (e.g., UNIXkernel). In some implementations, the operating system 1052 can includeinstructions for performing voice authentication. For example, operatingsystem 1052 can implement security lockout and voice authenticationfeatures.

The memory 1050 can also store communication instructions 1054 tofacilitate communicating with one or more additional devices, one ormore computers and/or one or more servers. The memory 1050 can includegraphical user interface instructions 1056 to facilitate graphic userinterface processing; sensor processing instructions 1058 to facilitatesensor-related processing and functions; phone instructions 1060 tofacilitate phone-related processes and functions; electronic messaginginstructions 1062 to facilitate electronic-messaging related processesand functions; web browsing instructions 1064 to facilitate webbrowsing-related processes and functions; media processing instructions1066 to facilitate media processing-related processes and functions;GNSS/Navigation instructions 1068 to facilitate GNSS andnavigation-related processes and functions; and/or camera instructions1070 to facilitate camera-related processes and functions.

The memory 1050 can store other software instructions 1072 to facilitateother processes and functions, such as security and/or authenticationprocesses and functions. For example, the software instructions caninclude instructions for performing voice authentication on perapplication or per feature basis and for allowing a user to configureauthentication requirements of each application or feature available ona device.

The memory 1050 can also store other software instructions (not shown),such as web video instructions to facilitate web video-related processesand functions; and/or web shopping instructions to facilitate webshopping-related processes and functions. In some implementations, themedia processing instructions 1066 are divided into audio processinginstructions and video processing instructions to facilitate audioprocessing-related processes and functions and video processing-relatedprocesses and functions, respectively. An activation record andInternational Mobile Equipment Identity (IMEI) 1074 or similar hardwareidentifier can also be stored in memory 1050.

Each of the above identified instructions and applications cancorrespond to a set of instructions for performing one or more functionsdescribed above. These instructions need not be implemented as separatesoftware programs, procedures, or modules. The memory 1050 can includeadditional instructions or fewer instructions. Furthermore, variousfunctions of the computing device 1000 can be implemented in hardwareand/or in software, including in one or more signal processing and/orapplication specific integrated circuits.

As described above, one aspect of the present technology is thegathering and use of data available from various sources to improve thedelivery to users of invitational content or any other content that maybe of interest to them. The present disclosure contemplates that in someinstances, this gathered data may include personal information data thatuniquely identifies or can be used to contact or locate a specificperson. Such personal information data can include demographic data,location-based data, telephone numbers, email addresses, home addresses,or any other identifying information.

The present disclosure recognizes that the use of such personalinformation data, in the present technology, can be used to the benefitof users. For example, the personal information data can be used todeliver targeted content that is of greater interest to the user.Accordingly, use of such personal information data enables calculatedcontrol of the delivered content. Further, other uses for personalinformation data that benefit the user are also contemplated by thepresent disclosure.

The present disclosure further contemplates that the entitiesresponsible for the collection, analysis, disclosure, transfer, storage,or other use of such personal information data will comply withwell-established privacy policies and/or privacy practices. Inparticular, such entities should implement and consistently use privacypolicies and practices that are generally recognized as meeting orexceeding industry or governmental requirements for maintaining personalinformation data private and secure. For example, personal informationfrom users should be collected for legitimate and reasonable uses of theentity and not shared or sold outside of those legitimate uses. Further,such collection should occur only after receiving the informed consentof the users. Additionally, such entities would take any needed stepsfor safeguarding and securing access to such personal information dataand ensuring that others with access to the personal information dataadhere to their privacy policies and procedures. Further, such entitiescan subject themselves to evaluation by third parties to certify theiradherence to widely accepted privacy policies and practices.

Despite the foregoing, the present disclosure also contemplatesembodiments in which users selectively block the use of, or access to,personal information data. That is, the present disclosure contemplatesthat hardware and/or software elements can be provided to prevent orblock access to such personal information data. For example, in the caseof advertisement delivery services, the present technology can beconfigured to allow users to select to “opt in” or “opt out” ofparticipation in the collection of personal information data duringregistration for services. In another example, users can select not toprovide location information for targeted content delivery services. Inyet another example, users can select to not provide precise locationinformation, but permit the transfer of location zone information.

Therefore, although the present disclosure broadly covers use ofpersonal information data to implement one or more various disclosedembodiments, the present disclosure also contemplates that the variousembodiments can also be implemented without the need for accessing suchpersonal information data. That is, the various embodiments of thepresent technology are not rendered inoperable due to the lack of all ora portion of such personal information data. For example, content can beselected and delivered to users by inferring preferences based onnon-personal information data or a bare minimum amount of personalinformation, such as the content being requested by the deviceassociated with a user, other non-personal information available to thecontent delivery services, or publically available information.

What is claimed is:
 1. A method comprising: receiving, by one or moreprocessors, a first set of trace data from a first mobile deviceoperating in an environment, the first set of trace data describing afirst set of states of a first trajectory of the first mobile device inthe environment, the first set of trace data including a first set ofwireless access point data and a first set of sensor measurementscollected by the first mobile device while operating in the environment,the first set of sensor measurements including pressure measurements;receiving, by the one or more processors, a second set of trace datafrom a second mobile device operating in the environment, the second setof trace data describing a second set of states of a second trajectoryof the second mobile device in the environment, the second set of tracedata including a second set of wireless access point data and a secondset of sensor measurements collected by the second mobile device whileoperating in the environment, the second set of sensor measurementsincluding pressure measurements; determining, by the one or moreprocessors, floor transitions in the first trajectory and the secondtrajectory based on the pressure measurements; dividing, by the one ormore processors, the first and second trajectories into floor nodesbased on the floor transitions; determining, by the one or moreprocessors, a distance between the floor nodes; determining, by the oneor more processors, a floor ordinal based on the distance; comparing, bythe one or more processors, portions of the first and secondtrajectories having the same floor ordinal using an optimization solverand a number of location constraints; and generating, by the one or moreprocessors, a radio map based on results of the comparing.
 2. The methodof claim 1, further comprising: determining, by the one or moreprocessors, a first location constraint based on the first and secondset of sensor measurements; determining, by the one or more processors,a second location constraint based on a similarity between the first andsecond sets of wireless access point data; and generating, by the one ormore processors, a radio map based on the first and second locationconstraints.
 3. The method of claim 2, further comprising: determining,by the one or more processors, a third location constraint based onlocation fixes; and generating, by the one or more processors, a radiomap based on the first, second and third location constraints.
 4. Themethod of claim 2, wherein the first and second location constraints areposterior probabilities computed from the first and second sets ofsensor measurements or the first and second sets of wireless accesspoint data.
 5. The method of claim 2, wherein the first or second setsof sensor measurements include pedometer data.
 6. The method of claim 2,wherein determining the similarity between the first and second sets ofwireless access point data further comprises: grouping the first andsecond sets of wireless access point data into a time window; computinga distance between the first and second sets of wireless access pointdata; comparing the distance with a threshold value; and generating thesecond location constraint based on a result of the comparing.
 7. Themethod of claim 6, wherein the distance is a computed by combining aJaccard distance and a Euclidean distance.
 8. The method of claim 1,wherein determining a second location constraint based on signalstrength measurements in the first and second sets of wireless accesspoint data, further comprises: comparing the signal strengthmeasurements to a threshold value; and generating the second locationconstraint based on a result of the comparing.
 9. The method of claim 1,wherein the first and second sets of trace data each include at leastlatitude, longitude, and heading.
 10. A system comprising: one or moreprocessors; memory storing instructions that when executed by the one ormore processors, cause the one or more processors to perform operationscomprising: receiving a first set of trace data from a first mobiledevice operating in an environment, the first set of trace datadescribing a first set of states of a first trajectory of the firstmobile device in the environment, the first set of trace data includinga first set of wireless access point data and a first set of sensormeasurements collected by the first mobile device while operating in theenvironment, the first set of sensor measurements including pressuremeasurements; receiving a second set of trace data from a second mobiledevice operating in the environment, the second set of trace datadescribing a second set of states of a second trajectory of the secondmobile device in the environment, the second set of trace data includinga second set of wireless access point data and a second set of sensormeasurements collected by the second mobile device while operating inthe environment, the second set of sensor measurements includingpressure measurements; determining floor transitions in the firsttrajectory and the second trajectory based on the pressure measurements;dividing the first and second trajectories into floor nodes based on thefloor transitions; determining a distance between the floor nodes;determining a floor ordinal based on the distance; comparing portions ofthe first and second trajectories having the same floor ordinal using anoptimization solver and a number of location constraints; and generatinga radio map based on results of the comparing.
 11. The system of claim10, further comprising: determining, by the one or more processors, afirst location constraint based on the first and second set of sensormeasurements; determining, by the one or more processors, a secondlocation constraint based on a similarity between the first and secondsets of wireless access point data; and generating, by the one or moreprocessors, a radio map based on the first and second locationconstraints.
 12. The system of claim 11, further comprising:determining, by the one or more processors, a third location constraintbased on location fixes; and generating, by the one or more processors,a radio map based on the first, second and third location constraints.13. The system of claim 12, wherein the first and second locationconstraints are posterior probabilities computed from the first andsecond sets of sensor measurements or the first and second sets ofwireless access point data.
 14. The system of claim 12, wherein thefirst or second sets of sensor measurements include pedometer data. 15.The system of claim 12, wherein determining the similarity between thefirst and second sets of wireless access point data further comprises:grouping the first and second sets of wireless access point data into atime window; computing a distance between the first and second sets ofwireless access point data; comparing the distance with a thresholdvalue; and generating the second location constraint based on a resultof the comparing.
 16. The system of claim 15, wherein the distance is acomputed by combining a Jaccard distance and a Euclidean distance. 17.The system of claim 10, wherein determining a second location constraintbased on signal strength measurements in the first and second sets ofwireless access point data, further comprises: comparing the signalstrength measurements to a threshold value; and generating the secondlocation constraint based on a result of the comparing.
 18. The systemof claim 10, wherein the first and second sets of trace data eachinclude at least latitude, longitude, and heading.
 19. A non-transitory,computer-readable storage medium having instructions stored thereon,that when executed by one or more processors, cause the one or moreprocessors to perform operations comprising: receiving a first set oftrace data from a first mobile device operating in an environment, thefirst set of trace data describing a first set of states of a firsttrajectory of the first mobile device in the environment, the first setof trace data including a first set of wireless access point data and afirst set of sensor measurements collected by the first mobile devicewhile operating in the environment, the first set of sensor measurementsincluding pressure measurements; receiving a second set of trace datafrom a second mobile device operating in the environment, the second setof trace data describing a second set of states of a second trajectoryof the second mobile device in the environment, the second set of tracedata including a second set of wireless access point data and a secondset of sensor measurements collected by the second mobile device whileoperating in the environment, the second set of sensor measurementsincluding pressure measurements; determining floor transitions in thefirst trajectory and the second trajectory based on the pressuremeasurements; dividing the first and second trajectories into floornodes based on the floor transitions; determining a distance between thefloor nodes; determining a floor ordinal based on the distance;comparing portions of the first and second trajectories having the samefloor ordinal using an optimization solver and a number of locationconstraints including the floor ordinal; and generating a radio mapbased on results of the comparing.
 20. The non-transitory,computer-readable storage medium of claim 19, the operations furthercomprising: determining, by the one or more processors, a first locationconstraint based on the first and second set of sensor measurements; anddetermining, by the one or more processors, a second location constraintbased on a similarity between the first and second sets of wirelessaccess point data.